Image processing of aerial imagery for energy infrastructure analysis using joint image identification

ABSTRACT

A computer-implemented method for processing images to identify Energy Infrastructure (EI) features within aerial images of global terrain is provided. The image processing method identifies information about EI features by applying an EI feature recognition model to aerial images of global terrain. The EI feature recognition model identifies the EI feature information according to image content of the aerial image. The method further provides updates to the identification of the EI feature information according to relationships between identified EI features.

RELATED APPLICATIONS

This application claims the benefit of prior U.S. Provisional PatentApplication Ser. No. 62/792,368, filed Jan. 14, 2019, and U.S.Provisional Patent Application Ser. No. 62/811,864, filed Feb. 28, 2019,both of which are hereby incorporated by reference in its entirety forall purposes.

TECHNICAL FIELD

The present invention generally relates to image processing systems andmethods for processing aerial imagery. The present invention furtherrelates to identifying, based on image processing, the presence andstatus of terrestrial energy infrastructure related features and forproviding the information thereby obtained to users of an onlineplatform.

BACKGROUND

Infrastructure of various differing types is required to harvest energyfrom natural resources, such as hydrocarbons (for example oil and gas),solar radiation, wind and hydroelectric sources. For a given energyresource type, constituent components of the infrastructure (hereontermed “Energy Infrastructure features” and abbreviated to “EIfeatures”) are often proximally-located to one another within the samelocale or geographical region, for example at an oilfield site, a solarpower station, a wind farm or a hydroelectric station.

As one example, hydraulic fracturing, or fracking, is a process specificto the hydrocarbon energy industry wherein a hydraulic liquid such aswater or gel is injected into shale rock under pressure in order tocreate or expand cracks to facilitate the extraction of subterraneannatural gas and oil. Use of this technique has grown rapidly in recentyears.

Water is not only needed to initiate the fracturing process (theinjectate), but may also often be recovered, produced or released aspart of the operation. This water may be a return of the injected wateror may be underground water that is released as a result of thefracturing. The quantity of the returned water can often be large, forexample, exceeding by far the quantity of oil obtained from the well.

The nature of the fracturing process therefore brings about arequirement not only to source large amounts of water at the outset of aproject, but also to dispose-of or treat and recycle water during theproject or upon its completion. Transportation of water from source tosite, or between sites, can incur significant costs and thereby reducethe available margin for profit during production. Such costs may bemitigated by identifying and selecting water source, disposal ortreatment options that are geographically local to the fracturing site,or which exploit efficient water transport infrastructure such aspipeline networks.

In support of this need for efficient water management in the energyindustry, tools to facilitate a dynamic online platform for watersourcing, recycling and disposal may be employed in which buyers andsellers of water source or disposal capacity may exchange informationrelated to either an availability-of or a requirement-for water,including a number of relevant attributes such as its quantity,location, type, and quality.

Such a platform may be further extended to address not only the waterresource needs associated with oilfield exploration and development, butalso the need and supply of other associated resources, services, orinfrastructure.

In further extensions, such a platform may be applied to energyindustries other than oil and gas, for example to renewable energysources such as solar, wind and hydroelectric. By means of example, theplatform may be used to provide information regarding the status of adeveloping solar power station, wind farm or hydroelectric site, inorder that users of the platform, such as suppliers of associatedservices, equipment or infrastructure, are timely-informed of upcomingopportunities and may take the information into account in theircommercial planning.

Accordingly, there is therefore a need for a more timely, efficient,reliable, automated and cost-reduced identification of energyinfrastructure features and determination of energy infrastructure sitestatus.

SUMMARY

A method is described for processing images to identify EnergyInfrastructure (EI) features. The method comprises receiving at leastone aerial image of a portion of global terrain, and applying an EIfeature recognition model to the image to identify a first EI feature ata first confidence level according to image content of the image. Themethod associates the first EI feature with a first EI feature type anda first location within the portion of the global terrain. The methodretrieves an identification of a second EI feature in the portion of theglobal terrain and associates the second EI feature with a second EIfeature type and a second location within the global terrain. The methodupdates the identification of the first EI feature to a secondconfidence level based at least on the first confidence level, theidentification of the second EI feature, and information on arelationship between the first EI feature and the second EI feature.

Other aspects and advantages of the invention will become apparent fromthe following detailed description, taken in conjunction with theaccompanying drawings, which illustrate by way of example the principlesof the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram depicting an embodiment of a networkenvironment comprising client devices in communication with onlineplatform devices, public records information source devices, aerialimage source devices, and server devices, according to some embodiments.

FIG. 1B is a block diagram depicting a cloud computing environmentcomprising client devices, for example user device and subscriberdevice, in communication with cloud service providers, according to someembodiments.

FIGS. 1C and 1D are block diagrams depicting embodiments of computingdevices useful in connection with the methods and systems describedherein.

FIG. 2A shows a system suitable for processing aerial images and forproviding EI feature and EI site status information to an onlineplatform, according to some embodiments.

FIG. 2B shows an example configuration of a system for processing aerialimages and for providing EI feature and EI site status information to anonline platform, according to some embodiments.

FIG. 3A shows a simplified block diagram of a system for training an EIfeature recognition model in a training mode, according to someembodiments.

FIG. 3B shows a simplified block diagram of a system for identifying EIfeatures using an EI feature recognition model in a working mode,according to some embodiments.

FIG. 3C shows a simplified block diagram of a system showing an EIfeature recognition model in a training mode and a working mode,according to some embodiments.

FIG. 4 illustrates an example of a method for processing images toidentify EI features within aerial images of global terrain, accordingto some embodiments.

FIG. 5A shows a simplified block diagram of a system for identifying afirst EI feature using an EI feature recognition model and informationon a second EI feature, according to some embodiments.

FIG. 5B shows a simplified block diagram of a system for identifying anEI feature using two EI feature recognition models, according to someembodiments.

FIG. 6 illustrates an example of a method for processing images toidentify EI features, according to some embodiments.

FIG. 7A shows an example of a system for selecting and storing workingand training aerial images to apply to an EI feature recognition model,according to some embodiments.

FIG. 7B shows an example of a system for selecting aerial images toapply to an EI feature recognition model, according to some embodiments.

FIG. 8 illustrates an example of a method for selecting aerial imagesfor image processing to identify EI features, according to someembodiments.

FIG. 9A illustrates oilfield-related EI features associated with an EIsite, according to some embodiments.

FIG. 9B illustrates solar-related EI features associated with an EIsite, according to some embodiments.

FIG. 9C shows a simplified block diagram of a system for determining acomposite indication of EI site status, according to some embodiments.

FIG. 9D shows a simplified block diagram of an alternative system fordetermining a composite indication of EI site status, according to someembodiments.

FIG. 10 illustrates an example of a method for processing images todetermine EI site status, according to some embodiments.

FIG. 11 shows examples of preconfigured and bespoke report typesprovided by an online platform, according to some embodiments.

DETAILED DESCRIPTION

The utility of an online platform relating to energy infrastructure (EI)and associated resources or services is naturally reliant upon thetimely availability of accurate and up-to-date information regarding thestatus of the EI development or activity in a given geographical area.Not only is such information required as an input to drive the platform,it is also more generally of significant interest to a variety ofindustry players. For example, in the context of the oil and gasindustry, information on where drilling is occurring or is about tooccur, and which parties are involved, may be used by suppliers ofresources, equipment and services to identify upcoming customeropportunities, by financial analysts and institutions to predict marketdynamics, or by oilfield operators for competitive intelligencepurposes.

The timely and accurate identification of EI features and their currentstatus is fundamental in enabling the above. However, regions in whichsuch EI features may exist (for example areas of oilfield exploration),often span large geographical areas, leading to correspondingly largecosts and effort to maintain associated data that is both timely andaccurate. For example, in the oil and gas industry, identifyingfrac-water pits (a storage pit for water), their water levels andsurface owners via manual investigation such as driving to sites andperforming visual inspections is time intensive and the data becomesquickly outdated. An alternative option is to perform human analysis ofsuitable aerial imagery of the relevant area, for example usingsatellite image data. However, these processes are againmanually-intensive, time consuming, and costly, leading to outdated andpotentially-erroneous data. Furthermore, satellite images of sufficientresolution and quality for a human to unequivocally detect a feature ina single review are prohibitively costly. Yet further alternativemethods may attempt to track upcoming EI activity (such as thecommencement of oilfield drilling) solely via permit data fromgovernment agencies. However, some regions may not have permitrequirements for certain types of energy resource exploration or EIdevelopment, and even where permit data is available, this is oftenalready outdated by the time it becomes of public record. The disclosureherein provides a technical solution to these problems and describes asystem and methods for timely, efficient, reliable, automated andcost-reduced identification of EI features and status. The technicalsolutions described herein further provide novel image processingmethods and techniques tailored to the identification of EI features andstatus.

For purposes of reading the description of the various embodimentsbelow, the following descriptions of the sections of the specificationsand their respective contents may be helpful:

Section A describes a network environment and computing environmentwhich may be useful for practicing embodiments described herein.

Section B describes embodiments of systems and methods that provide thetechnical solution of r timely, efficient, reliable, automated andcost-reduced identification of EI features and status.

A. Computing and Network Environment

Prior to discussing specific embodiments of the present solution, it maybe helpful to describe aspects of the operating environment as well asassociated system components (e.g., hardware elements) in connectionwith the methods and systems described herein. Referring to FIG. 1A, anembodiment of a network environment is depicted. In a brief overview,the network environment may include one or more clients 102 a-102 n(also generally referred to as local machines(s) 102, client(s) 102,client node(s) 102, client machine(s) 102, client computer(s) 102,client device(s) 102, endpoint(s) 102, or endpoint node(s) 102) incommunication with one or more servers 106 a-106 n (also generallyreferred to as server(s) 106, node(s) 106, machine(s) 106, or remotemachine(s) 106), one or more online platforms 180 a-180 n (alsogenerally referred to as online platforms(s) 180, platform node(s) 180,platform machine(s) 180, or remote online platform machine(s) 180), oneor more public records information source 150 a-150 n (also generallyreferred to as public records information source(s) 150, record node(s)150, record machine(s) 150, or remote record machine(s) 150), and one ormore aerial image sources 101 a-101 n (also generally referred to asaerial information source(s) 101, image source(s) 101, image sourcemachine(s) 101, or remote image source machine(s) 101) via one or morenetworks 104. In some embodiments, one or more of client 102, onlineplatform 180, or public records information source 150 has the capacityto function as both a node seeking access to resources provided by aserver and as a server providing access to hosted resources for otherclients 102 a-102 n, online platforms 180 a-180 n, and public recordsinformation sources 150 a-150 n. Examples of client(s) 102 includesuser(s) 190 and subscriber(s) 195.

Although FIG. 1A shows a network 104 between clients 102, onlineplatforms 180, public records information source 150, aerial imagesource 101 and the servers 106, in examples clients 102, onlineplatforms 180, public records information source 150, aerial imagesource 101 and servers 106 may be on the same network 104. In someembodiments, there are multiple networks 104 between clients 102, onlineplatforms 180, public records information source 150, aerial imagesource 101 and the servers 106. In one of these embodiments, a network104′ (not shown) may be a private network and a network 104 may be apublic network. In another of these embodiments, a network 104 may be aprivate network and a network 104′ may be a public network. In stillanother of these embodiments, networks 104 and 104′ may both be privatenetworks. Servers 106 may be used to generically refer to all of onlineplatforms 180, public records information source 150, aerial imagesource 101, and servers 106. Clients 102, online platforms 180, andpublic records information source 150 may process input from server 106and/or may provide access as needed to various applications, modules,and other software components of server 106 to other variousapplications, modules, and other software components of server 106.

The network 104 may be connected via wired or wireless links. Wiredlinks may include Digital Subscriber Line (DSL), coaxial cable lines, oroptical fiber lines. Wireless links may include Bluetooth®, BluetoothLow Energy (BLE), ANT/ANT+, ZigBee, Z-Wave, Thread, Wi-Fi®, WorldwideInteroperability for Microwave Access (WiMAX®), mobile WiMAX®,WiMAX®-Advanced, NFC, SigFox, LoRa, Random Phase Multiple Access (RPMA),Weightless-N/P/W, an infrared channel or a satellite band. The wirelesslinks may also include any cellular network standards to communicateamong mobile devices, including standards that qualify as 2G, 3G, 4G, or5G. The network standards may qualify as one or more generations ofmobile telecommunication standards by fulfilling a specification orstandards such as the specifications maintained by the InternationalTelecommunication Union. The 3G standards, for example, may correspondto the International Mobile Telecommuniations-2000 (IMT-2000)specification, and the 4G standards may correspond to the InternationalMobile Telecommunication Advanced (IMT-Advanced) specification. Examplesof cellular network standards include AMPS, GSM, GPRS, UMTS, CDMA2000,CDMA-1×RTT, CDMA-EVDO, LTE, LTE-Advanced, LTE-M1, and Narrowband IoT(NB-IoT). Wireless standards may use various channel access methods,e.g., FDMA, TDMA, CDMA, or SDMA. In some embodiments, different types ofdata may be transmitted via different links and standards. In otherembodiments, the same types of data may be transmitted via differentlinks and standards.

The network 104 may be any type and/or form of network. The geographicalscope of the network may vary widely and the network 104 can be a bodyarea network (BAN), a personal area network (PAN), a local-area network(LAN), e.g., Intranet, a metropolitan area network (MAN), a wide areanetwork (WAN), or the Internet. The topology of the network 104 may beof any form and may include, e.g., any of the following: point-to-point,bus, star, ring, mesh, or tree. The network 104 may be an overlaynetwork which is virtual and sits on top of one or more layers of othernetworks 104′. The network 104 may be of any such network topology asknown to those ordinarily skilled in the art capable of supporting theoperations described herein. The network 104 may utilize differenttechniques and layers or stacks of protocols, including, e.g., theEthernet protocol, the internet protocol suite (TCP/IP), the ATM(Asynchronous Transfer Mode) technique, the SONET (Synchronous OpticalNetworking) protocol, or the SDH (Synchronous Digital Hierarchy)protocol. The TCP/IP internet protocol suite may include applicationlayer, transport layer, internet layer (including, e.g., IPv4 and IPv4),or the link layer. The network 104 may be a type of broadcast network, atelecommunications network, a data communication network, or a computernetwork.

In some embodiments, the system may include multiple, logically-groupedservers 106. In one of these embodiments, the logical group of serversmay be referred to as a server farm or a machine farm. In another ofthese embodiments, the servers 106 may be geographically dispersed. Inother embodiments, a machine farm may be administered as a singleentity. In still other embodiments, the machine farm includes aplurality of machine farms. The servers 106 within each machine farm canbe heterogeneous—one or more of the servers 106 or machines 106 canoperate according to one type of operating system platform (e.g.,Windows, manufactured by Microsoft Corp. of Redmond, Wash.), while oneor more of the other servers 104 can operate according to another typeof operating system platform (e.g., Unix, Linux, or Mac OSX).

In one embodiment, servers 106 in the machine farm may be stored inhigh-density rack systems, along with associated storage systems, andlocated in an enterprise data center. In this embodiment, consolidatingthe servers 106 in this way may improve system manageability, datasecurity, the physical security of the system, and system performance bylocating servers 106 and high-performance storage systems on localizedhigh-performance networks. Centralizing the servers 106 and storagesystems and coupling them with advanced system management tools allowsmore efficient use of server resources.

The servers 106 of each machine farm do not need to be physicallyproximate to another server 106 in the same machine farm. Thus, thegroup of servers 106 logically grouped as a machine farm may beinterconnected using a wide-area network (WAN) connection or ametropolitan-area network (MAN) connection. For example, a machine farmmay include servers 106 physically located in different continents ordifferent regions of a continent, country, state, city, campus, or room.Data transmission speeds between servers 104 in the machine farm can beincreased if the servers 106 are connected using a local-area network(LAN) connection or some form of direct connection. Additionally, aheterogeneous machine farm may include one or more servers 106 operatingaccording to a type of operating system, while one or more other serversexecute one or more types of hypervisors rather than operating systems.In these embodiments, hypervisors may be used to emulate virtualhardware, partition physical hardware, virtualize physical hardware, andexecute virtual machines that provide access to computing environments,allowing multiple operating systems to run concurrently on a hostcomputer. Native hypervisors may run directly on the host computer.Hypervisors may include VMware ESX/ESXi, manufactured by VMWare, Inc.,of Palo Alta, Calif.; the Xen hypervisor, an open source product whosedevelopment is overseen by Citrix Systems, Inc. of Fort Lauderdale,Fla.; the HYPER-V hypervisors provided by Microsoft, or others. Hostedhypervisors may run within an operating system on a second softwarelevel. Examples of hosted hypervisors may include VMWare Workstation andVirtualBox, manufactured by Oracle Corporation of Redwood City, Calif.

Management of the machine farm may be de-centralized. For example, oneor more servers 106 may comprise components, subsystems and modules tosupport one or more management services for the machine farm. In one ofthese embodiments, one or more servers 106 provide functionality formanagement of dynamic data, including techniques for handling failover,data replication, and increasing the robustness of the machine farm.Each server 106 may communicate with a persistent store and, in someembodiments, with a dynamic store.

Server 106, online platform 180, public records information source 150,and aerial image source 101 may be a file server, application server,web server, proxy server, appliance, network appliance, gateway, gatewayserver, virtualization server, deployment server, SSL VPN server, orfirewall. In one embodiment, a plurality of servers 106, onlineplatforms 180, public records information sources 150, and aerial imagesources 101 may be in the path between any two communicating servers106, online platforms 180, public records information sources 150, oraerial image sources 101.

Referring to FIG. 1B, a cloud computing environment is depicted. A cloudcomputing environment may provide user 190 and subscriber 195 with oneor more resources provided by a network environment. The cloud computingenvironment may include one or more users 190 a-190 n and one or moresubscribers 195 a-195 n in communication with the cloud 108 over one ormore networks 104. Users 190 and subscribers 195 may include, e.g.,thick clients, thin clients, and zero clients. A thick client mayprovide at least some functionality even when disconnected from thecloud 108 or servers 106. A thin client or zero client may depend on theconnection to the cloud 108 or server 106 to provide functionality. Azero client may depend on the cloud 108 or other networks 104 or servers106 to retrieve operating system data for user 190 or subscriber 195.The cloud 108 may include back end platforms, e.g., servers 106,storage, server farms or data centers.

The cloud 108 may be public, private, or hybrid. Public clouds mayinclude public servers 106 that are maintained by third parties toclient(s) 102, for example user(s) 190 and subscriber(s) 195 or ownersof client(s) 102, user(s) 190, and/or subscriber(s) 195. The servers 106may be located off-site in remote geographical locations as disclosedabove or otherwise. Public clouds may be connected to the servers 106over a public network. Private clouds may include private servers 106that are physically maintained by client(s) 102, for example user(s) 190and/or subscriber(s) 195 or owners of client(s) 102, user(s) 190, and/orsubscriber(s) 195. Private clouds may be connected to the servers 106over a private network 104. Hybrid clouds may include both private andpublic networks 104 and servers 106.

Cloud 108 may also include a cloud-based delivery, e.g., Software as aService (SaaS) 110, Platform as a Service (PaaS) 112, and Infrastructureas a Service (IaaS) 114. IaaS may refer to a user renting the user ofinfrastructure resources that are needed during a specified time period.IaaS provides may offer storage, networking, servers or virtualizationresources from large pools, allowing the users to quickly scale up byaccessing more resources as needed. Examples of IaaS include Amazon WebServices (AWS) provided by Amazon, Inc. of Seattle, Wash., RackspaceCloud provided by Rackspace Inc. of San Antonio, Tex., Google ComputeEngine provided by Google Inc. of Mountain View, Calif., or RightScaleprovided by RightScale, Inc. of Santa Barbara, Calif. PaaS providers mayoffer functionality provided by IaaS, including, e.g., storage,networking, servers or virtualization, as well as additional resources,e.g., the operating system, middleware, or runtime resources. Examplesof PaaS include Windows Azure provided by Microsoft Corporation ofRedmond, Wash., Google App Engine provided by Google Inc., and Herokuprovided by Heroku, Inc. of San Francisco Calif. SaaS providers mayoffer the resources that PaaS provides, including storage, networking,servers, virtualization, operating system, middleware, or runtimeresources. In some embodiments, SaaS providers may offer additionalresources including, e.g., data and application resources. Examples ofSaaS include Google Apps provided by Google Inc., Salesforce provided bySalesforce.com Inc. of San Francisco, Calif., or Office365 provided byMicrosoft Corporation. Examples of SaaS may also include storageproviders, e.g., Dropbox provided by Dropbox Inc. of San Francisco,Calif., Microsoft OneDrive provided by Microsoft Corporation, GoogleDrive provided by Google Inc., or Apple iCloud provided by Apple Inc. ofCupertino, Calif.

Client(s) 102, for example user(s) 190 and/or subscriber(s) 195 mayaccess IaaS resources with one or more IaaS standards, including, e.g.,Amazon Elastic Compute Cloud (EC2), Open Cloud Computing Interface(OCCI), Cloud Infrastructure Management Interface (CIMI), or OpenStackstandards. Some IaaS standards may allow clients access to resourcesover HTTP and may use Representational State Transfer (REST) protocol orSimple Object Access Protocol (SOAP). Client(s) 102, for example user(s)190 and/or subscriber(s) 195 may access PaaS resources with differentPaaS interfaces. Some PaaS interfaces use HTTP packages, standard JavaAPIs, JavaMail API, Java Data Objects (JDO), Java Persistence API (JPA),Python APIs, web integration APIs for different programming languagesincluding, e.g., Rack for Ruby, WSGI for Python, or PSGI for Perl, orother APIs that may be built on REST, HTTP, XML, or other protocols.Client(s) 102, for example user(s) 190 and/or subscriber(s) 195 mayaccess SaaS resources through the use of web-based user interfaces,provided by a web browser (e.g., Google Chrome, Microsoft InternetExplorer, or Mozilla Firefox provided by Mozilla Foundation of MountainView, Calif.). Client(s) 102, for example user(s) 190 and/orsubscriber(s) 195 may also access SaaS resources through smartphone ortablet applications, including e.g., Salesforce Sales Cloud, or GoogleDrive App. Client(s) 102, for example user(s) 190 and/or subscriber(s)195 may also access SaaS resources through the client operating system,including e.g., Windows file system for Dropbox.

In some embodiments, access to IaaS, PaaS, or SaaS resources may beauthenticated. For example, a server or authentication server mayauthenticate a user via security certificates, HTTPS, or API keys. APIkeys may include various encryption standards such as, e.g., AdvancedEncryption Standard (AES). Data resources may be sent over TransportLayer Security (TLS) or Secure Sockets Layer (SSL).

Client(s) 102, for example user(s) 190 and/or subscriber(s) 195 andserver 106 may be deployed as and/or executed on any type and form ofcomputing device, e.g., a computer, network device or appliance capableof communicating on any type and form of network and performing theoperations described herein.

FIGS. 1C and 1D depict block diagrams of a computing device 100 usefulfor practicing an embodiment of the client 102, online platform 180,public records information source 150, aerial image source 101 and theserver 106. As shown in FIGS. 1C and 1D, each computing device 100includes a central processing unit 133, and a main memory unit 134. Asshown in FIG. 1C, a computing device 100 may include a storage device128, an installation device 116, a network interface 118, and 110controller 123, display devices 124 a-124 n, a keyboard 126 and apointing device 127, e.g., a mouse. The storage device 128 may include,without limitation, an operating system 129, software 131, and asoftware of a feature recognition system 121. As shown in FIG. 1D, eachcomputing device 100 may also include additional optional elements,e.g., a memory port 103, a bridge 171, one or more input/output devices132 a-132 n (generally referred to using reference numeral 132), and acache memory 141 in communication with the central processing unit 133.

The central processing unit 133 is any logic circuitry that responds toand processes instructions fetched from the main memory unit 134. Inmany embodiments, the central processing unit 133 is provided by amicroprocessor unit, e.g.: those manufactured by Intel Corporation ofMountain View, Calif.; those manufactured by Motorola Corporation ofSchaumburg, Ill.; the ARM processor and TEGRA system on a chip (SoC)manufactured by Nvidia of Santa Clara, Calif.; the POWER4 processor,those manufactured by International Business Machines of White Plains,N.Y.; or those manufactured by Advanced Micro Devices of Sunnyvale,Calif. The computing device 100 may be based on any of these processors,or any other processor capable of operating as described herein. Thecentral processing unit 133 may utilize instruction level parallelism,thread level parallelism, different levels of cache, and multi-coreprocessors. A multi-core processor may include two or more processingunits on a single computing component. Examples of multi-core processorsinclude the AMD PHENOM IIX2, INTER CORE i5 and INTEL CORE i4.

Main memory unit 134 may include on or more memory chips capable ofstoring data and allowing any storage location to be directly accessedby the microprocessor 133. Main memory unit 134 may be volatile andfaster than storage 128 memory. Main memory units 134 may be DynamicRandom-Access Memory (DRAM) or any variants, including staticRandom-Access Memory (SRAM), Burst SRAM or SynchBurst SRAM (BSRAM), FastPage Mode DRAM (FPM DRAM), Enhanced DRAM (EDRAM), Extended Data OutputRAM (EDO RAM), Extended Data Output DRAM (EDO DRAM), Burst Extended DataOutput DRAM (BEDO DRAM), Single Data Rate Synchronous DRAM (SDR SDRAM),Double Data Rate SDRAM (DDR SDRAM), Direct Rambus DRAM (DRDRAM), orExtreme Data Rate DRAM (XDR DRAM). In some embodiments, the main memory134 or the storage 128 may be non-volatile, e.g., non-volatile readaccess memory (NVRAM), flash memory non-volatile static RAM (nvSRAM),Ferroelectric RAM (FeRAM), Magnetoresistive RAM (MRAM), Phase-changememory (PRAM), conductive-bridging RAM (CBRAM),Silicon-Oxide-Nitride-Oxide-Silicon (SONOS), Resistive RAM (RRAM),Racetrack, Nano-RAM (NRAM), or Millipede memory. The main memory 134 maybe based on any of the above described memory chips, or any otheravailable memory chips capable of operating as described herein. In theembodiment shown in FIG. 1C, the processor 133 communicates with mainmemory 134 via a system bus 151 (described in more detail below). FIG.1D depicts an embodiment of a computing device 100 in which theprocessor communicates directly with main memory 134 via a memory port103. For example, in FIG. 1D the main memory 134 may be DRDRAM.

FIG. 1D depicts an embodiment in which the main processor 133communicates directly with cache memory 141 via a secondary bus,sometimes referred to as a backside bus. In other embodiments, the mainprocessor 133 communicates with cache memory 141 using the system bus151. Cache memory 141 typically has a faster response time than mainmemory 134 and is typically provided by SRAM, BSRAM, or EDRAM. In theembodiment shown in FIG. 1D, the processor 133 communicates with variousI/O devices 132 via a local system bus 151. Various buses may be used toconnect the central processing unit 133 to any of the I/O devices 132,including a PCI bus, a PCI-X bus, or a PCI-Express bus, or a NuBus. Forembodiments in which the I/O device is a video display 124, theprocessor 133 may use an Advanced Graphic Port (AGP) to communicate withthe display 124 or the I/O controller 123 for the display 124. FIG. 1Ddepicts an embodiment of a computer 100 in which the main processor 133communicates directly with I/O device 312 b or other processors 133′ viaHYPERTRANSPORT, RAPIDIO, or INFINIBAND communications technology. FIG.1D also depicts an embodiment in which local busses and directcommunication are mixed: the processor 133 communicates with I/O device132 a using a local interconnect bus while communicating with I/O device132 b directly.

A wide variety of I/O devices 132 a-132 n may be present in thecomputing device 100. Input devices may include keyboards, mice,trackpads, trackballs, touchpads, touch mice, multi-touch touchpads andtouch mice, microphones, multi-array microphones, drawing tablets,cameras, single-lens reflex cameras (SLR), digital SLR (DSLR), CMOSsensors, accelerometers, infrared optical sensors, pressure sensors,magnetometer sensors, angular rate sensors, depth sensors, proximitysensors, ambient light sensors, gyroscopic sensors, or other sensors.Output devices may include video displays, graphical displays, speakers,headphones, inkjet printers, laser printers, and 3D printers.

Devices 132 a-132 n may include a combination of multiple input oroutput (I/O) devices, including, e.g., Microsoft KINECT, NintendoWiimote for the WII, Nintendo WII U GAMEPAD, or Apple iPhone. Some I/Odevices 132 a-132 n allow gesture recognition inputs through combiningsome of the inputs and outputs. Some I/O devices 132 a-132 n provide forfacial recognition which may be utilized as an input for differentpurposes including authentication and other commands. Some I/O devices132 a-132 n provide for voice recognition and inputs, including, e.g.,Microsoft KINECT, SIRI for iPhone by Apple, Google Now or Google VoiceSearch, and Alexa by Amazon.

Additional I/O devices 132 a-132 n have both input and outputcapabilities, including, e.g., haptic feedback devices, touchscreendisplays, or multi-touch displays. Touchscreen, multi-touch displays,touchpads, touch mice, or other touch sensing devices may use differenttechnologies to sense touch, including, e.g., capacitive, surfacecapacitive, projected capacitive touch (PCT), in cell capacitive,resistive, infrared, waveguide, dispersive signal touch (DST), in-celloptical, surface acoustic wave (SAW), bending wave touch (BWT), orforce-based sensing technologies. Some multi-touch devices may allow twoor more contact points with the surface, allowing advanced functionalityincluding, e.g., pinch, spread, rotate, scroll, or other gestures. Sometouchscreen devices, including, e.g., Microsoft PIXELSENSE orMulti-Touch Collaboration Wall, may have larger surfaces, such as on atable-top or on a wall, and may also interact with other electronicdevices. Some I/O devices 132 a-132 n, display devices 124 a-124 n orgroup of devices may be augmented reality devices. The I/O devices maybe controlled by an I/O controller 123 as shown in FIG. 1C. The I/Ocontroller may control one or more I/O devices, such as, e.g., akeyboard 126 and a pointing device 127, e.g., a mouse or optical pen.Furthermore, an I/O device may also provide storage and/or aninstallation device 116 for the computing device 100. In still otherembodiments, the computing device 100 may provide USB connections (notshown) to receive handheld USB storage devices. In further embodiments,a I/O device 132 may be a bridge between the system bus 151 and anexternal communication bus, e.g., a USB bus, a SCSI bus, a FireWire bus,an Ethernet bus, a Gigabit Ethernet bus, a Fiber Channel bus, or aThunderbolt bus.

In some embodiments, display devices 124 a-124 n may be connected to I/Ocontroller 123. Display devices may include, e.g., liquid crystaldisplays (LCD), thin film transistor LCD (TFT-LCD), blue phase LCD,electronic papers (e-ink) displays, flexile displays, light emittingdiode displays (LED), digital light processing (DLP) displays, liquidcrystal on silicon (LCOS) displays, organic light-emitting diode (OLED)displays, active-matrix organic light-emitting diode (AMOLED) displays,liquid crystal laser displays, time-multiplexed optical shutter (TMOS)displays, or 3D displays. Examples of 3D displays may use, e.g.,stereoscopy, polarization filters, active shutters, or auto stereoscopy.Display devices 124 a-124 n may also be a head-mounted display (HMD). Insome embodiments, display devices 124 a-124 n or the corresponding I/Ocontrollers 123 may be controlled through or have hardware support forOPENGL or DIRECTX API or other graphics libraries.

In some embodiments, the computing device 100 may include or connect tomultiple display devices 124 a-124 n, which each may be of the same ordifferent type and/or form. As such, any of the I/O devices 132 a-132 nand/or the I/O controller 123 may include any type and/or form ofsuitable hardware, software, or combination of hardware and software tosupport, enable or provide for the connection and use of multipledisplay devices 124 a-124 n by the computing device 100. For example,the computing device 100 may include any type and/or form of videoadapter, video card, driver, and/or library to interface, communicate,connect or otherwise use the display devices 124 a-124 n. In oneembodiment, a video adapter may include multiple connectors to interfaceto multiple display devices 124 a-124 n. In other embodiments, thecomputing device 100 may include multiple video adapters, with eachvideo adapter connected to one or more of the display devices 124 a-124n. In some embodiments, any portion of the operating system of thecomputing device 100 may be configured for using multiple displays 124a-124 n. In other embodiments, one or more of the display devices 124a-124 n may be provided by one or more other computing devices 100 a or100 b connected to the computing device 100, via the network 104. Insome embodiments, software may be designed and constructed to useanother computer's display device as a second display device 124 a forthe computing device 100. For example, in one embodiment, an Apple iPadmay connect to a computing device 100 and use the display of the device100 as an additional display screen that may be used as an extendeddesktop. One ordinarily skilled in the art will recognize and appreciatethe various ways and embodiments that a computing device 100 may beconfigured to have multiple display devices 124 a-124 n.

Referring again to FIG. 1C, the computing device 100 may comprise astorage device 128 (e.g., one or more hard disk drives or redundantarrays of independent disks) for storing an operating system or otherrelated software, and for storing application software programs such asany program related to the feature recognition system software 121.Examples of storage device 128 include, e.g., hard disk drive (HDD);optical drive including CD drive, DVD drive, or BLU-RAY drive;solid-state drive (SSD); USB flash drive; or any other device suitablefor storing data. Some storage devices may include multiple volatile andnon-volatile memories, including, e.g., solid state hybrid drives thatcombine hard disks with solid state cache. Some storage device 128 maybe non-volatile, mutable, or read-only. Some storage device 128 may beinternal and connect to the computing device 100 via a bus 151. Somestorage device 128 may be external and connect to the computing device100 via an I/O device 132 that provides an external bus. Some storagedevice 128 may connect to the computing device 100 via the networkinterface 118 over a network 104, including, e.g., the Remote Disk forMACBOOK AIR by Apple. Some client devices 100 may not require anon-volatile storage device 128 and may be thin clients or zero clients102. Some storage device 128 may also be used as an installation device116 and may be suitable for installing software and programs.Additionally, the operating system and the software can be run from abootable medium, for example, a bootable CD, e.g., KNOPPIX, a bootableCD for GNU/Linux that is available as a GNU/Linux distribution fromknoppix.net.

Client device 100 may also install software or application from anapplication distribution platform. Examples of application distributionplatforms include the App Store for iOS provided by Apple, Inc., the MacApp Store provided by Apple, Inc., GOOGLE PLAY for Android OS providedby Google Inc., Chrome Webstore for CHROME OS provided by Google Inc.,and Amazon Appstore for Android OS and KINDLE FIRE provided byAmazon.com, Inc. An application distribution platform may facilitateinstallation of software on a client device 102. An applicationdistribution platform may include a repository of applications on aserver 106 or a cloud 108, which the clients 102 a-102 n may access overa network 104. An application distribution platform may includeapplication developed and provided by various developers. A user of aclient device 102 may select, purchase and/or download an applicationvia the application distribution platform.

Furthermore, the computing device 100 may include a network interface118 to interface to the network 104 through a variety of connectionsincluding, but not limited to, standard telephone lines LAN or WAN links(e.g., 802.11, T1, T3, Gigabit Ethernet, InfiniBand), broadbandconnections (e.g., ISDN, Frame Relay, ATM, Gigabit Ethernet,Ethernet-over-SONET, ADSL, VDSL, BPON, GPON, fiber optical includingFiOS), wireless connections, or some combination of any or all of theabove. Connections can be established using a variety of communicationprotocols (e.g., TCP/IP, Ethernet, ARCNET, SONET, SDH, Fiber DistributedData Interface (FDDI), IEEE 802.11a/b/g/n/ac CDMA, GSM, WiMAX and directasynchronous connections). In one embodiment, the computing device 100communicates with other computing devices 100′ via any type and/or formof gateway or tunneling protocol e.g., Secure Socket Layer (SSL) orTransport Layer Security (TLS), or the Citrix Gateway Protocolmanufactured by Citrix Systems, Inc. The network interface 118 maycomprise a built-in network adapter, network interface card, PCMCIAnetwork card, EXPRESSCARD network card, card bus network adapter,wireless network adapter, USB network adapter, modem or any other devicesuitable for interfacing the computing device 100 to any type of networkcapable of communication and performing the operations described herein.

A computing device 100 of the sort depicted in FIGS. 1C and 1D mayoperate under the control of an operating system, which controlsscheduling of tasks and access to system resources. The computing device100 can be running any operating system such as any of the versions ofthe MICROSOFT WINDOWS operating systems, the different releases of theUnix and Linux operating systems, any version of the MAC OS forMacintosh computers, any embedded operating system, any real-timeoperating system, any open source operating system, any proprietaryoperating system, any operating systems for mobile computing devices, orany other operating system capable of running on the computing deviceand performing the operations described herein. Typical operatingsystems include, but are not limited to: WINDOWS 2000, WINDOWS Server2012, WINDOWS CE, WINDOWS Phone, WINDOWS XP, WINDOWS VISTA, and WINDOWS4, WINDOWS RT, WINDOWS 8 and WINDOW 10, all of which are manufactured byMicrosoft Corporation of Redmond, Wash.; MAC OS and iOS, manufactured byApple, Inc.; and Linux, a freely-available operating system, e.g., LinuxMint distribution (“distro”) or Ubuntu, distributed by Canonical Ltd. ofLondon, United Kingdom; or Unix or other Unix-like derivative operatingsystems; and Android, designed by Google Inc., among others. Someoperating systems, including, e.g., the CHROME OS by Google Inc., may beused on zero clients or thin clients, including, e.g., CHROMEBOOKS.

The computer system 100 can be any workstation, telephone, desktopcomputer, laptop or notebook computer, netbook, ULTRABOOK, tablet,server, handheld computer, mobile telephone, smartphone or otherportable telecommunications device, media playing device, a gamingsystem, mobile computing device, or any other type and/or form ofcomputing, telecommunications or media device that is capable ofcommunication. The computer system 100 has sufficient processor powerand memory capacity to perform the operations described herein. In someembodiments, the computing device 100 may have different processors,operating systems, and input devices consistent with the device. TheSamsung GALAXY smartphones, e.g., operate under the control of Androidoperating system developed by Google, Inc. GALAXY smartphones receiveinput via a touch interface.

In some embodiments, the computing device 100 is a gaming system. Forexample, the computer system 100 may comprise a PLAYSTATION 3, orPERSONAL PLAYSTATION PORTABLE (PSP), or a PLAYSTATION VITA devicemanufactured by the Sony Corporation of Tokyo, Japan, or a NINTENDO DS,NINTENDO 3DS, NINTENDO WII, or a NINTENDO WII U device manufactured byNintendo Co., Ltd., of Kyoto, Japan, or an XBOX 340 device manufacturedby Microsoft Corporation.

In some embodiments, the computing device 100 is a digital audio playersuch as the Apple IPOD, IPOD Touch, and IPOD NANO lines of devices,manufactured by Apple Computer of Cupertino, Calif. Some digital audioplayers may have other functionality, including, e.g., a gaming systemor any functionality made available by an application from a digitalapplication distribution platform. For example, the IPOD Touch mayaccess the Apple App Store. In some embodiments, the computing device100 is a portable media player or digital audio player supporting fileformats including, but not limited to, MP3, WAV, M9A/AAC, WMA ProtectedAAC, AIFF, Audible audiobook, Apple Lossless audio file formats and.mov, .m4v, and .mp4 MPEG-4 (H.244/MPEG-4 AVC) video file formats.

In some embodiments, the computing device 100 is a tablet e.g., the IPADline of devices by Apple; GALAXY TAB family of devices by Samsung; orKINDLE FIRE, by Amazon.com, Inc. of Seattle, Wash. In other embodiments,the computing device 100 is an eBook reader, e.g., the KINDLE family ofdevices by Amazon.com, or NOOK family of devices by Barnes & Noble, Inc.of New York City, N.Y.

In some embodiments, client 102 includes a combination of devices, e.g.,a smartphone combined with a digital audio player or portable mediaplayer. For example, one of these embodiments is a smartphone, e.g., theiPhone family of smartphones manufactured by Apple, Inc.; a SamsungGALAXY family of smartphones manufactured by Samsung, Inc; or a MotorolaDROID family of smartphones. In yet another embodiment, client 102 is alaptop or desktop computer equipped with a web browser and a microphoneand speaker system, e.g., a telephony headset. In these embodiments, theclient(s) 102 are web-enabled and can receive and initiate phone calls.In some embodiments, a laptop or desktop computer is also equipped witha webcam or other video capture device that enables video chat and videocall.

In some embodiments, the status of one or more machines 102, 106 in thenetwork 104 is monitored, generally as part of network management. Inone of these embodiments, the status of a machine may include anidentification of load information (e.g., the number of processes on themachine, CPU and memory utilization), of port information (e.g., thenumber of available communication ports and the port addresses), or ofsession status (e.g., the duration and type of processes, and whether aprocess is active or idle). In another of these embodiments, thisinformation may be identified by a plurality of metrics, and theplurality of metrics can be applied at least in part towards decisionsin load distribution, network traffic management, and network failurerecovery as well as any aspects of operations of the present solutiondescribed herein. Aspects of the operating environments and componentsdescribed above will become apparent in the context of the systems andmethods disclosed herein.

B. Timely, Efficient, Reliable, Automated and Cost-ReducedIdentification of Energy Infrastructure (EI) Features and Status Systemsand Methods

The following describes systems and methods that are useful for timely,efficient, reliable, automated and cost-reduced identification of EIfeatures and status. Timely and accurate identification of EI featuresand their status is fundamental in enabling a platform for the exchangeof resources and services necessary for EI development and energyharvesting or production.

The term EI “feature” is used extensively in this disclosure to refer toa constituent component of energy infrastructure. For example, in thecontext of the oil and gas industry, an EI feature may be a component ofinfrastructure at an existing or potential drilling or hydraulicfracturing oilfield site, whereas in the context of the solar industry,an EI feature may be a solar panel, or a solar panel array at a solarpower station. An EI “site”, such as an existing or potential drillingor hydraulic fracturing oilfield site, or a solar or wind-power station,may comprise a concentration or a plurality of EI features within agiven locale, for example, each serving a purpose or function to enableoperations at that site. An EI feature is associated with a geographicallocation at which it is located and may further be associated with an EIfeature type and/or an EI feature status attribute. EI features ofnumerous types are discussed within this disclosure. In the context ofthe oil and gas industry, EI feature types may include for example, awell pad, a frac-water pit, a rig, a tank battery and so forth. In thecontext of the solar energy industry, EI feature types may include forexample, a solar panel array, a power conditioning substation, or asecurity fence. Further examples of EI feature types may be found inTable 2. EI feature status attributes may be indicative of a status of aparticular aspect of an EI feature at the associated geographicallocation. Numerous EI feature status attributes are also discussedwithin this disclosure, and may include for example, an area or size, afluid level or volume, a number of sub-parts, an ownership and so forth.Further examples of EI feature status attributes may be found in Table3. In the context of this disclosure, an “EI feature” may be identifiedas the result (i.e., using the output-of) of an image processingoperation. The image processing operation may further classify anidentified EI feature as being of a particular EI feature type.

Aspects of this disclosure also relate to image processing usingArtificial Intelligence (AI). The term “feature” as used in the AI imageprocessing discipline may sometimes be used in reference to particularimage content within an input image to be processed. Whilst such an“input image content feature” may, in the context of this disclosure, berelated to an “EI feature” (for example, an input image may containimage content depicting an oilfield well pad), the two terms are notsynonymous as the former may relate to a feature within images input toan AI image processing function, whilst the latter may refer to acomponent of energy infrastructure at an EI site that may be identifiedas a result of an AI image processing function.

System Overview

In a general overview, FIG. 2A shows a system 200 which is configured touse image processing to automatically scan aerial images captured byaerial image source 101 to identify EI features and EI site activity orstatus, and which is configured for presenting the information therebyobtained to users 190 and subscribers 195 of online platform 180. Aerialimage source 101 may be a satellite or any other suitable overheadimaging device.

Referring to FIG. 2A in more detail, in some embodiments, system 200 mayinclude image selection processing manager 212. Image selectionprocessing manager 212 may be configured to select one or more aerialimages of a portion of global terrain from a set of available aerialimages, and to provide the selected images (such as those shown asselected image(s) 215 in FIG. 2B) to, for example, EI featuredetermination manager 280, for onward processing. Image selectionprocessing manager 212 may further be configured to store selectedimages in image selection storage 290.

The set of available aerial images from which image selection processingmanager 212 may select may comprise any suitable aerial images, such asmay be obtained from aerial image source 101, aerial image storage 105,or image selection storage 290. Image selection processing manager 212may be configured to communicate either directly, or over a network 104,with aerial image storage 105, aerial image source 101 or imageselection storage 290. Image selection processing manager 212 may alsobe configured to communicate with EI feature determination manager 280.

Images stored by image selection processing manager 212, for example inimage selection storage 290, may also be subdivided into training imagesand working images. Other subdivisions, groupings, labelling orcategorizations of the images selected or stored by image selectionprocessing manager 212 are also possible, such as may be based on forexample, location, image resolution, image format, image capture deviceand so on. However, for ease of explanation, image selection processingmanager 212 may in general be configured to provide one or more trainingimage 310 and/or one more working image 312 to other components ofserver 106, such images sourced from any suitable aerial image sourcesuch as aerial image source 101, aerial image storage 105, or imageselection storage 290. Image selection processing manager 212 may be anapplication, service, daemon, routine, or other executable logic forselecting, analyzing, and processing aerial images.

EI feature determination manager 280 may be configured to process aerialimages, such as those selected by image selection processing manager 212or those stored in image selection storage 290, in order to determineinformation on EI features. In doing so, EI feature determinationmanager 280 may be configured to comprise one or more EI featurerecognition models 216, such as those depicted in FIG. 2A as 216 a, 216b, . . . 216 n. In examples, EI feature determination manager 280 maycommunicate with image selection processing manager 212, EI featurerelationship manager 220, EI status determination manager 222, imageselection storage 290, EI feature database 230, public recordsinformation source 250 or relationship manager storage 292.

In some examples, EI feature recognition model 216 may communicate withEI feature recognition model training manager 214, image selectionprocessing manager 212, joint classification processing manager 218, EIfeature relationship manager 220, EI status determination manager 222,image selection storage 290, EI feature database 230, public recordsinformation source 250 or relationship manager storage 292. Inembodiments, EI feature recognition model 216 may be configured to useartificial intelligence based on, for example, a neural network or othertype of machine learning. In examples, EI feature determination manager280 or EI feature recognition model 216 may be configured to combineartificial intelligence and image processing with information from oneor more supplemental information sources 270, to identify for example i)the presence (or change thereof) of an EI feature, and itsclassification or type or ii) one or more status attributes (or changethereof) associated with an EI feature. Such identified information istermed hereon EI ‘feature-level’ information (for example, as shown asEI feature-level information 282 in FIG. 2B) and may be stored in EIfeature database 230 or provided to online platform 180, for example,via network 104. EI feature-level information may include, for example,information about a presence, type, or status of an identified EIfeature (e.g., an oilfield frac-water pit or drilling rig, see Table 2for further examples). The supplemental information from the one or moresupplemental information sources 270, and which be used to assist in theidentification of the EI feature information, may include non-imagebased information about aspects of an EI site or EI feature.

One or more EI feature recognition models 216 may be configured tooperate in training mode or working mode. In training mode, the EIfeature recognition model 216 may be supplied with one or more trainingimages 310 (for example, from image selection processing manager 212)and with supplemental information that may be known a-priori. Suchsupplemental information may be obtained from an EI feature database 230or from public records information source 250. In working mode, the EIfeature recognition mode 216 may be supplied with one or more workingimages 312 (for example, from image selection processing manager 212).EI feature recognition model 216 may also be configured to usesupplemental information (such as from EI feature database 230 or frompublic records information source 250) during working mode to improveits accuracy and its ability to recognize EI features.

EI feature recognition model 216 may be an application, service, daemon,routine, or other executable logic for applying artificial intelligenceand/or machine learning to analyze data inputs to provide EIfeature-level information. In a training mode of operation, the EIfeature recognition model 216 may be trained to associate image contentwithin a training image with EI features and EI feature types known tobe present within the training image. Training images include aerialimages of global terrain that may contain identified EI features ofidentified EI feature types. In the training mode of operation, the EIfeature recognition model 216 may establish associations between imagecontent within the training image, EI features and EI feature types. Theimage content may comprise pixel values (e.g., color, hue, intensity,saturation, etc.) or other image-based informational content encoded orotherwise represented within the training image. In a working mode ofoperation, the EI feature recognition model 216 may then be configuredfor application to one or more working images to identify EI featuresand EI feature types within the working image. Working images includeaerial images of global terrain that may contain unidentified EIfeatures of EI feature types. The EI feature recognition model 216 isconfigured to apply, to the working images, its previously-learnedassociations between image content, EI features and EI feature types.Based on image content of the working images, the EI feature recognitionmodel 216 identifies EI features and EI feature types. In some examples,EI feature recognition model 216 is configured to provide a confidencelevel associated with one or more components of the EI feature-levelinformation obtained as a result of its operation. A confidence levelmay represent a surety or confidence of the model in the identificationof the EI feature-level information. For example, a confidence level mayindicate a percentage confidence that an identification is correct.

In some examples, system 200 may include EI feature recognition modeltraining manager 214, for example comprised within EI featuredetermination manager 280. EI feature recognition model training manager214 may communicate with image selection processing manager 212 and EIfeature recognition model 216 and may have access to EI feature datarecords 240 which may be stored in EI feature database 230. EI featurerecognition model training manager 214 may have access to one or moresupplemental information sources 270. In embodiments, EI featurerecognition model training manager 214 may have access to public records260 which may be stored in public records information source 250, and/oraccess to EI feature data records 240 which may be stored in EI featuredatabase 230. EI feature recognition model training manager 214 maycommunicate with EI status determination manager 222 and may shareinformation with online platform 180 via network 104.

In embodiments, EI feature recognition model training manager 214 may beconfigured to train one or more EI feature recognition models 216 a, 216b . . . 216 n. Artificial intelligence models may be of a variety oftypes, for example linear regression models, logistic regression models,linear discriminant analysis models, decision tree models, naïve bayesmodels, K-nearest neighbors models, learning vector quantization models,support vector machines, bagging and random forest models, and deepneural networks. In general, all AI models aim to learn a function whichprovides the most precise correlation between input values (X) andoutput values (Y):

Y=f(X)

In general, EI feature recognition model training manager 214 trains oneor more EI feature recognition models 216 a, 216 b . . . 216 n usinghistoric sets of inputs (X) and outputs (Y) that are known to becorrelated. For example, in a linear regression AI model represented bythe expression:

Y=B ₀ ×B ₁ X

A set of n historical data points (X_(i),Y_(i)) are used to estimate thevalues for B₀ and B₁, for example:

${{B_{1} = \frac{\sum_{i = 1}^{n}\left( {\left( {X_{i} - {\overset{¯}{X}}_{i}} \right) \times \left( {Y_{i} - {\overset{\_}{Y}}_{i}} \right)} \right)}{\sum_{i = 1}^{n}\left( \left( {X_{i} - {\overset{¯}{X}}_{i}} \right)^{2} \right.}}B_{0}} = {\overset{¯}{Y_{i}} - {B_{1}\left( {\overset{¯}{X}}_{i} \right)}}$

Parameters B₀ and B₁ may be considered coefficients of the AI model. Themodel with these initial coefficients is then used to predict the outputof the model Y_(i,M) given the set of historic inputs X_(i). Thus,Y_(i,M) corresponds to a derived output of the model given X_(i), andmay differ to a known (or “correct”) output for input X_(i). The errorof these predictions may be calculated using Root Mean Square Error(RMSE), for example;

${RMSE} = \sqrt{\sum\limits_{i = 1}^{n}\;\frac{\left( {Y_{i,M} - Y_{i}} \right)^{2}}{n}}$

EI feature recognition model training manager 214 then adjusts thecoefficients B₀ and B₁ to minimize the RMSE over multiple historicaldata sets (X_(i), Y_(i)). Different types of AI models use differenttechniques to adjust the weights (or values) of various coefficients, ingeneral by using historical data sets that are correlated in the waythat the model is trying to predict in new data sets by minimizing thepredicted error of the model when applied to the historical data sets.

EI feature recognition model training manager 214 may be an application,service, daemon, routine, or other executable logic for creating,training, and/or updating EI feature recognition models.

In training mode, EI feature recognition model training manager 214 maybe configured to use training images, for example from image selectionprocessing manager 212 or image selection storage 290, and supplementalinformation from one or more supplemental information source(s) 270 tocreate, update or modify EI feature recognition model 216. In someexamples, training images are identified through finding a current EIfeature of interest (e.g., one for which its presence is certain) in aportion of global terrain. Historical images of the same area of terrainare gathered from aerial image storage 105. These historical images arethen used as training images. In some examples, EI feature recognitionmodel training manager 214 may be configured to use historical images ofa known EI feature to update a structure, weights, coefficients,parameters, variables, thresholds or probabilities associated with aneural network associated with EI feature recognition model 216. Forexample, when the presence of an EI feature is confirmed, for example bya site visit, a permit, or other method with a high degree of certainty,EI feature recognition model training manager 214 may obtain historicalimages of the same portion of global terrain in which the EI feature wasconfirmed in order to update the model. In some examples, EI featurerecognition model training manager 214 may be configured to update amachine learning algorithm associated with EI feature recognition model216. EI feature recognition model 216 obtained by use of EI featurerecognition model training manager 214 may be configured to subsequentlyoperate in working mode, for example to identify and classify EIfeatures within one or more working images.

In some examples, system 200 may include joint classification processingmanager 218, for example comprised within EI feature determinationmanager 280. Joint classification processing manager 218 may beconfigured to process information on both a first and a second EIfeature together with information on a relationship between the firstand second EI features, in order to identify or improve anidentification of the first EI feature.

In some examples, joint classification processing manager 218 may beconfigured to communicate with one or more EI feature recognition models216 a, 216 b, . . . 216 n, with EI feature relationship manager 220,with relationship manager storage 292, and/or with EI feature database230. In some examples, joint classification processing manager may beconfigured to communicate EI feature-level information obtained as aresult of its operation to EI feature database 230 for subsequentstorage.

Joint classification processing manager 218 may be an application,service, daemon, routine, or other executable logic for identifying EIfeatures.

System 200 may be configured to comprise one or more supplementalinformation source 270 comprising supplemental information. In examples,supplemental information may comprise EI feature information, such asmay, for example, be stored within EI feature database 230 in the formof one or more EI feature data records 240. In other examples,supplemental information may comprise one or more public records 260,such as may, for example, be stored within a public records informationsource 250. Whilst the term ‘public’ is used within system 200description in relation to items 250 and 260, it shall be appreciatedthat this is an exemplar and does not limit the scope of the descriptionto public availability of the information contained therein, and thatother equivalent or similar information sources, whether private orpublic, freely available or available at a cost, may be equally suitablefor use within system 200. More generally, examples of supplementalinformation types that may be obtained from one or more supplementalinformation sources 270 and which may be used by system 200 are given inTable 1.

TABLE 1 Examples of Supplemental Information Oilfield Drilling permitsor permit filings, spud reports, and Permits completion reports for oiland gas wells, water wells and disposal wells Easements Easement andpermit records related to public and private land and road crossingsBuilding Permits or consents for EI feature construction permits or fornon-EI features Energy Development applications, plans, consents,consultation Infrastructure reports, environmental study reports andother development documentation associated with Energy Infrastructureproposals, development consents or reports Maps Pipeline maps, roadmaps,stormwater management maps Deeds Surface (land) ownership and leaserecords Prospective Announcements of the availability of public orprivate land land for sale, or applications to acquire public or privatetransactions land Land use Proposals or consents for change of land useproposals or consents Mineral Rights Mineral (subterranean) ownershipand lease records Contracts Commercial contract announcements, biddinginvitation announcements, information on energy supply agreements Newsand News feeds, recruitment announcements, marketplace Media listingsSeismology Seismology records and well logs Information Field Data Datacollected from the field (such as from phone calls, site visits andreports) Weather Data Current, historical or forecast weather conditionsat a location Legal Legal notices, court filings information GPS andTruck or truck driver GPS coordinates from a tracking Location device orphone Information Sensor Data Data from sensors to monitor any componentof energy infrastructure or to provide any other or potentially- relatedinformation such as seismic activity, weather data, or the location ormovement of vehicles, equipment, resources or supplies HydrologicalHydrological studies and maps of surface water flows Information andgroundwater aquifers

System 200 may be configured such that supplemental informationsource(s) 270 may communicate with, connect to, or be accessed by othercomponents of the system, for example, any of image selection processingmanager 212, EI feature determination manager 280, EI featurerecognition model training manager 214, joint classification processingmanager 218, EI feature relationship manager 220, EI statusdetermination manager 222, image selection storage 290 and relationshipmanager storage 292.

In a further optional aspect, system 200 may comprise EI statusdetermination manager 222, which may be configured to determine acomposite indication of EI site status (for example, as shown ascomposite indication of EI site status 284 in FIG. 2B). EI statusdetermination manager 222 may be configured to use, combine or aggregateone or more or a plurality of EI feature-level information 282 (in someexamples stored in EI feature database 230 as EI feature data records240) and supplemental information from supplemental informationsource(s) 270 (in some examples using either EI feature data records 240or public records 260) in order to determine or predict activity relatedto energy infrastructure at a larger scale, for example activity at anEI site that encompasses multiple EI features. For example, EI statusdetermination manager 222 may combine information, such as thedetermination of the presence of a clearing, a service road, and afrac-water pit all within a radius of 500 meters of a particulargeographical location within an oilfield region, to generate a compositeindication of EI site status 284 that an oilfield exploration site isunderdevelopment at that location. In another example, a compositeindication of EI site status 284 may be, for example, that an oilfielddrilling site has developed to a stage at which all resources andcomponents necessary to commence drilling are present, and hencedrilling is likely to occur at the site within a specified timescale.The resources and components necessary to commence drilling may includefor example, a frac-water pit, a drilling rig, pipeline infrastructureand tank batteries, and these may, in some examples, have beenpreviously and individually identified at the EI feature level. In afurther example, composite indication of EI site status 284 may be thatconstruction of an electrical substation has been identified within 1000m of a solar panel mounting frame arrangement spanning 5 hectares ofland, thus indicating an increased level of site completion, and a needfor solar panel installation services. Composite indication of EI sitestatus 284 may be communicated to online platform 180 either directly,or via a network 104.

In some examples of system 200, EI status determination manager 222 maybe configured to communicate with public records information source 250(for example directly, or via a network 104), with EI feature database230, with EI feature relationship manager 220, or with relationshipmanager storage 292.

EI status determination manager 222 may be an application, service,daemon, routine, or other executable logic for determining a compositeindication of EI site status based on information related to a pluralityof EI features.

In yet a further optional aspect, system 200 may comprise EI featurerelationship manager 220, which may be configured to determine and store(for example in relationship manager storage 292) relationships betweenEI features, EI feature types, EI feature attributes, EI feature status,supplemental information, public records, and so on. In some examples,relationships between EI features may be known a priori, for example, inan oilfield context, sand mines, trucks, and proppant stores arerelated, as the sand that is extracted from a sand mine is trucked to aproppant store. In other examples, relationships between EI features arelearned by EI feature relationship manager 220 over time, for example bydetermining that the appearance of two or more EI feature types arerelated. For example, EI feature relationship manager 220 may determinethat within an oilfield region, clearings of a particular size andorientation are related to well pad development, for example as aleading indicator. In examples, EI feature relationship manager 220 maydetermine that the presence of oilfield pipeline infrastructure combinedwith a frac-water pit and a well pad are indicative of the imminentappearance of oilfield drilling rigs. In some examples, EI featurerelationship manager 220 may comprise one or more neural network orother Artificial Intelligence and/or machine learning model configurableto learn inter-relationships between various EI features and aspects ofEI site development. In other examples, EI feature relationship manager220 may comprise pre-programmed or pre-configured inter-relationshipsbetween various EI features and aspects of EI site development.However-so-derived, the information on inter-relationships betweenvarious EI features and aspects of EI site development may be utilizedin some examples by EI feature determination manager (for example, byjoint classification processing manager 218) to improve the ability ofthe system to identify EI feature-level information 282, or by EI statusdetermination manager 222 to improve the ability of the system toidentify a composite indication of EI site status 284.

In some examples of system 200, EI feature relationship manager 220 maybe configured to communicate with EI status determination manager 222,with relationship manager storage 292, with EI feature determinationmanager 280, or with joint classification processing manager 218.

EI feature relationship manager 222 may be an application, service,daemon, routine, or other executable logic for determining or providinginformation on inter-relationships between various EI features andaspects of EI site development.

In some examples, system 200 may include online platform 180. Onlineplatform 180 may communicate with server 106, one or more users 190 orone or more subscribers 195 via one or more networks 104. In someembodiments, online platform 180 is arranged to receive EI feature-levelinformation 282, composite indication of EI site status 284, or bothfrom server 106. In examples, online platform may query server 106 toreceive EI feature-level information 282 and/or composite indication ofEI site status 284. In examples, server 106 may push EI feature-levelinformation 282 and/or composite indication of EI site status 284 toonline platform 180, for example when this information is updated or atpredetermined times or intervals. In some embodiments, online platform180 may be configured to access server 106 via an applicationprogramming interface (API). Access to server 106 by online platform 180may be controlled, for example access may require a subscription, ausername and password, or encryption keys. In some examples, informationmay be accessible on a free, paid or subscription basis, and may bepresented in its raw form, a distilled form, or in a manner that istailored to the specific requirements of—or format used by—the onlineplatform, for example in an appropriately-formatted document or report.Server 106 may be part of a cluster of servers 106. In some embodiments,tasks performed by server 106 may be performed by a plurality ofservers. These tasks may be allocated among the plurality of servers byan application, service, daemon, routine, or other executable logic fortask allocation. Server 106 may include a processor and memory. Some orall of server 106 may be hosted on cloud 108, for example by Amazon WebServices (AWS, Amazon Inc., Seattle, Wash.) or by Oracle Cloud (Oracle,Redwood City, Calif.).

In some embodiments, online platform is configured to provide EIfeature-level information 282, composite indication of EI site status284, or both, to users 190 or subscribers 195 of online platform 180.Access to online platform 180 by users 190 and/or subscribers 195 may becontrolled, for example access may require a subscription, a usernameand password, or encryption keys. In some examples, information may beaccessible on a free, paid or subscription basis, and may be presentedin its raw form, a distilled form, or in a manner that is tailored tothe specific requirements of, or format used by the users 190 and/orsubscribers 195, for example in an appropriately-formatted document orreport.

Any of online platform 180, image selection processing manager 212, EIfeature recognition model training manager 214, EI feature recognitionmodel 216, joint classification processing manager 218, EI featuredetermination manager 280, EI feature relationship manager 220, and EIstatus determination manager 222 may, for example, be a desktopcomputer, a laptop computer, a mobile device, a server, or any othersuitable computing device. Online platform 180, image selectionprocessing manager 212, EI feature recognition model training manager214, EI feature recognition model 216, joint classification processingmanager 218, EI feature determination manager 280, EI featurerelationship manager 220, and EI status determination manager 222 maycomprise a program, service, task, script, library, application or anytype and form of executable instructions or code executable on one ormore processors. Any of online platform 180, image selection processingmanager 212, EI feature recognition model training manager 214, EIfeature recognition model 216, joint classification processing manager218, EI feature determination manager 280, EI feature relationshipmanager 220, and EI status determination manager 222 may be combinedinto one or more modules, applications, programs, services, tasks,scripts, libraries, applications, or executable code.

In examples of system 200, the configuration and maintenance of server106 may be controlled by administrator 197 who may access or controlserver 106 either directly or via a network 104. System 200 may beconfigured such that operations that may be performed by administrator197 may include for example, the updating of software or firmware usedby any component of the server, the configuring, receiving or processingof diagnostic reports or logs, the configuring of parameters, variables,or thresholds used by any component of the server, the reading orwriting of data from any storage within server 106, or the performing ofa direct communication or a communication via a network 104 with systemcomponents external to server 106.

Per the description of FIG. 2A, system 200 may be flexibly configured ina multitude of different ways. In order to illustrate such possibilitiesfurther, and by means of example only, FIG. 2B shows one potentialconfiguration of system 200.

In FIG. 2B, aerial images are captured by aerial image source 101 andmay be optionally stored in aerial image storage 105. Such aerial imagesmay be provided to for example, image selection processing manager 212of server 106, through either a direct connection or via a network 104.Server 106 may be controlled, configured or administered byadministrator 197, who may be connected to server 106 directly or via anetwork 104. Image selection processing manager 212 may be configured toselect particular aerial images from either aerial image source 101 oraerial image storage 105 for onward processing. Image selectionprocessing manager 212 may optionally store selected images in, forexample, image selection storage 290. Selected image(s) 215 comprisingimages selected by image selection processing manager 212, may besubdivided into aerial images for EI feature recognition model trainingpurposes (hereon referred to as training images 310) and aerial imagesfor EI feature identification purposes (hereon referred to as workingimages 312). Image selection processing manager 212 may provide selectedaerial images 215 to EI feature determination manager 280, such imagescomprising for example, one or more training images 310 for EI featurerecognition model training purposes and/or one or more working images312 for EI feature identification purposes as shall further bedescribed.

In the example system configuration of FIG. 2B, EI feature determinationmanager 280 may be configured to receive aerial images from imageselection processing manager 212 or image selection storage 290, andoptionally, may also be configured to receive supplemental informationon EI features from one or more supplemental information sources 270.The information on EI features may comprise for example, one or more EIfeature data records 240 from an EI feature database 230, or one or morepublic records 260 from a public records information source 250. EIfeature determination manager 280 may be configured to operate on thereceived images and (if provided) on the supplemental information, inorder to identify or classify EI features and to thereby determine EIfeature-level information 282. EI feature determination manager 280 mayapply one or more EI feature recognition models 216 to the receivedimages to determine the EI feature-level information 282 by associatingimage content or other aspects of the received images with EI featuresor types. For example, EI feature determination manager 280 may apply anEI feature recognition model 216 a to detect a clearing and an EIfeature recognition model 216 b to detect a road to the same set ofreceived working images. In embodiments, EI feature determinationmanager 280 may receive supplemental information such as informationthat a permit to build a road within an oilfield region has beenrequested or granted, and based on the supplemental information, EIfeature determination manager may apply an EI feature recognition modelto a set of images to detect oilfield site activity that has arelationship with the presence of a road, for example the creation of aclearing. Optionally, to assist with this task, EI feature determinationmanager 280 may also comprise a joint classification processing manager218 that may be configured to utilize information received from EIfeature relationship manager 220. EI feature-level information 282 maybe provided to EI feature database 230 where it may be stored forexample in the form of one or more EI feature data records 240.Optionally, EI feature-level information 282 may also be provided toonline platform 180, for example directly, or via a network 104, forpotential onward use by users 190 or subscribers 195.

The example system configuration of FIG. 2B may further comprise an EIstatus determination manager 222 which may be configured to receive aplurality of EI feature-level information 282 such as may for example bestored in the form of EI feature data records 240 within EI featuredatabase 230. In one particular example, each of the plurality of EIfeature-level information 282 may relate to a different EI feature. EIstatus determination manager 222 may further be configured to receiveinformation on an inter-relation between EI features from EI featurerelationship manager 220 or relationship manager storage 292. EI statusdetermination manager 222 may further be configured to receivesupplemental information which may, for example, take the form of one ormore public records from a public records information source 250.

In the example system configuration of FIG. 2B, EI status determinationmanager 222 may operate to process the information received from EIfeature database 230, EI feature relationship manager 220 and optionallypublic records information source 250 to identify a composite indicationof EI site status 284. Composite indication of EI site status 284 may beprovided to online platform 180, for example directly, or via a network104, for potential onward use by users 190 or subscribers 195. Furtherdetails, embodiments and methods shall become apparent from thefollowing descriptions of additional figures.

EI Feature Classification

System 200 may be configured to recognize and classify EI features byprocessing aerial images of global terrain.

FIG. 3A illustrates an EI feature recognition model 216 in training modeof operation. One or more aerial image source(s) 101 may generate aerialimages of global terrain. One or more of such images may optionally bestored in aerial image storage 105 and provided (via optional network104) to image selection processing manager 212. Image selectionprocessing manager may store aerial images in image selection storage290. In some examples, a plurality of aerial images may be stored inimage selection storage 290 and be classified as training images 310.Image selection processing manager 212 or image selection storage 290may communicate with EI feature determination manager 280 or EI featurerecognition model 216. EI feature recognition model 216 may receive orretrieve information from one or more supplemental information source(s)270. EI feature recognition model 216 may access supplementalinformation sources 270 via a shared memory access, for example, or viaa network. In some examples, training mode of operation is managed by EIfeature recognition training manager 214. EI feature recognitiontraining manager 214 may for example select training images, retrievesupplemental information, and change parameters of the model such as astructure, weights, coefficients, parameters, variables, thresholds orprobabilities of EI feature recognition model.

In examples, applying the EI feature recognition model 216 includesrecognizing from an image time-sequence, a stage of EI featuredevelopment across the image time-sequence. In embodiments, EI featurerecognition model training manager 214 receives a training imagecomprising an image time-sequence, the image time-sequence including aplurality of aerial images of a portion of global terrain, each aerialimage of the plurality taken at one of a respective plurality of imagecapture times. EI feature recognition model 216 or EI featurerecognition model training manager 214 may receive, from a supplementalinformation source 270, information on EI features located within theportion of global terrain. In some examples, EI feature recognitionmodel training manager 214 may generate an EI feature recognition model216 according to the image time-sequence of the training image and theinformation on EI features located within the portion of global terrain,wherein the information on EI features located within the portion ofglobal terrain identifies the appearance, presence-of or recording-of anEI feature at a historical time that is between the earliest and thelatest of the plurality of image capture times. In some embodiments, EIfeature recognition model training manager 214 may identify a pattern ofstages of EI feature development across the time-sequence. Applying thetrained EI feature recognition model may include recognizing a stage ofEI feature development of the EI feature in the working image.

In some examples, EI feature recognition model 216 operating in atraining mode of operation obtains one or more pieces of supplementalinformation from one or more supplemental information sources 270.Supplemental information source 270 may comprise EI feature datadatabase 230 in which one or more EI data records 240 are stored. Inembodiments, the supplemental information source 270 may comprise publicrecords information source 250 in which one or more public records 260are stored.

EI feature recognition model 216 may comprise artificial intelligenceprocessing such as a neural network or other machine learning algorithm.During the training mode of operation, a structure, weights,coefficients, parameters, variables, thresholds or probabilities of theEI feature recognition model 216 are modified or updated by EI featurerecognition model training manager 214 in order that over one or moreiterations of such training, the ability of the EI feature recognitionmodel 216 to identify or recognize EI features, to classify EI featuresas belonging to a given EI feature type, and/or to determine a statusattribute of an EI feature, is improved. In this manner, and by usingthe information from supplemental information sources 270, system 200 isable to train one or more EI feature recognition models 216 withoutspecific need for human supervision.

In some examples, EI feature recognition model 216 may be trained torecognize one or more of the following EI feature types (Table 2) and toclassify these appropriately. The list is non-exhaustive.

TABLE 2 Examples of EI feature Types Frac-water pits These are typicallyman-made surface ponds or reservoirs used (also known as frac to storefresh or brackish water prior to its injection into an ponds or fracwater oilfield well, or flowback or produced water that may beimpoundments) returned from the oilfield well. Well pads Surface sitescomprised of a leveled, usually-rectangular area used to seat machineryand equipment for oilfield drilling, completing, producing andmaintaining oil and gas wells. Drilling rigs Machines and associatedstructures that perform the oilfield drilling. Pipeline Fixed ortemporary pipes to transport oil, gas or water. infrastructure Serviceroads Roads, typically unpaved, that enable transport of equipment andresources to and from Energy Infrastructure sites or features, forexample, in an oilfield context, to well pads or other oilfieldfacilities. Clearings Surface sites comprised of a leveled,often-rectangular area where the purpose of the site is not yet known.Trucks Vehicles used to transport resources, equipment and wasteproducts to and from EI sites or service locations. These may be used,for example, for sand, chemical, cement, water or oil transport. Tankbatteries A group of storage tanks connected to an oil or gas well orsaltwater- disposal well to receive oil or water produced by an oilfieldwell. Proppant stores Proppant is a solid material, usually sand, thatis injected into an oilfield well to help keep a newly-created fractureopen. An example of a proppant store would be a sand pile amassed at anoilfield well pad prior to commencement of drilling. Drilling reservepits Small reservoirs of water used for oilfield drilling muds and drillcuttings that are accumulated during oil and gas drilling operations.Frac spreads Large temporary gatherings of tanks, pressure pumps,trucks, pipes and other equipment and workers to fracture or “complete”a drilled oilfield well and thereby commence oil and or gas productionfrom the oilfield well. Sand mines An area where sand is extracted fromthe ground for use as proppant. Producing wells Drilled and completedoil and or gas wells that are now producing hydrocarbons. Flare systemsInfrastructure to facilitate the burning of gas or oil at an oilfieldsite, for example during well production testing, for safety oremergency purposes or to manage waste production. Flare systems maycomprise for example ground flares, elevated flares, flare stacks orassociated pipes. Solar panel mounts Fixed or sun-tracking solar panelmounting frames, or associated foundations, piles, ballasts, earthscrews or baseplates for the affixation of solar panels. Solar panelsPower generation infrastructure comprising a panel surface, or array ofpanel surfaces for the collection of radiant light energy and itsconversion to other forms, such as electricity Electrical Aninstallation or building comprising equipment for the substationsconditioning or adaptation of electrical power signals, voltages orwaveforms, for interconnection between power generation infrastructureand electrical power grids, or for power storage, metering, protectionor isolation. Security fences A fence to protect and secure energyinfrastructure sites or energy infrastructure features. BuildingsBuildings or offices to house components of energy infrastructure,communications, monitoring or other equipment, resources, supplies,vehicles, workers or staff. Cable systems Cable, mounts, pylons,trenches and associated infrastructure for overhead, ground-level orsubterranean transport of electrical energy. Wind Energy Powergeneration infrastructure associated with the collection of Collectorsenergy from wind and its conversion to other forms such as electricity,including tower structures, turbines, nacelles and rotors.Meteorological Infrastructure and sensors to enabling the monitoring ofMonitoring meteorological conditions and processing of the associateddata, Equipment comprising for example anemometers, thermometers,barometers, rain gauges, masts and data loggers. Construction Machineryor vehicles used to make clearings or to construct Equipment buildingsor infrastructure, comprising for example, forestry equipment, cranes,earth movers, excavators, borers, forklifts and trucks. Hydroelectric Abody of water used to drive hydroelectric turbines and generators.reservoirs or forebays Hydroelectric Structures to filter debris anddirect water from a hydroelectric intake structures forebay towards theturbine via a hydroelectric water conduit, channel or pipe (“penstock”).Penstocks Pipes to carry water between reservoirs, forebays, or intakestructures to a hydraulic turbine. Surge Chambers A tank used to controlpressure in a penstock. Hydroelectric Infrastructure to househydroelectric (hydraulic) turbines or Power Houses generators.Hydroelectric Areas used for the outflow return of water following itsuse by a Tailraces hydroelectric turbine.In some embodiments, EI feature recognition model 216 may be trained torecognize a developed or developing EI site, such as an oilfielddrilling or hydraulic fracturing site, as a whole, which may itselfcomprise one or more constituent EI features. In some embodiments, sucha developed or developing EI site may correspond to an additional EIfeature type of ‘an EI development site’.

EI Feature Status Attributes

Once an EI feature has been detected and classified as a given EIfeature type, EI feature recognition model 216 or EI featuredetermination manager 280 may also determine and track a status of oneor more attributes associated with that particular feature type. Thisprovides additional information of use to users of the online platform,or which may serve as a valuable input to other functions of the system(for example, the determination of composite indication of EI siteactivity or status).

Each EI feature type or classification may be associated with a suitableset of attributes. For example, a frac-water pit may have an attributeset comprising:

-   -   Area    -   Water level or volume    -   Water color, type or quality (for example, fresh water may be        clear or blue whereas brackish or ‘used’ water may be brown)    -   Connectivity (for example, whether pipelines to the frac-water        pit are visible)

Some of these attributes (such as ‘area’) would also be appropriate forother feature types, such as a well pad, whereas others (such as ‘waterlevel or volume’) would not. In general, feature types may each havedifferent attribute sets though some attributes may be common amongstfeature types.

The values of some of the attributes for a given feature may be derivedthrough suitable image processing, whereas the values of otherattributes may be derived using external data sources, such asgovernment or corporate records, or any information from public recordsinformation sources. For example, an area or location of a feature maybe derived from images, whilst the surface rights owner at that locationmay be derived using a land ownership database, this representing oneexample of a public records information source. For some attributetypes, it may also be possible to utilize other means of determiningstatus. For example, remote sensors may be deployed at sites to monitorwater levels, quality, salinity, or pH. Such remote sensors may reportvalues via a communication network to which the EI feature recognitionsystem or the online platform is connected.

For illustrative purposes, Table 3 below provides a non-exhaustive listof EI feature types and their attributes and example sources of data.The term “g/c records” represents “government or corporate records” inshorthand throughout.

TABLE 3 Examples of EI feature status attributes Feature Classification(Type) Attribute Example source of data Frac-water pit Detectiondate/time Image analysis Location Image analysis, g/c records Area orsize Image analysis, g/c records Surface rights owner Land ownershiprecords Mineral rights owner Land ownership records Fluid level orvolume Image analysis, sensor networks Fluid or material type Imageanalysis, sensor networks Fluid or material color Image analysis, sensornetworks Fluid or material Image analysis, sensor networks qualityMaterial or fluid Image analysis, sensor networks attribute ConnectivityImage analysis, g/c records Well pad Detection date/time Image analysisLocation Image analysis, g/c records Area or size Image analysis, g/crecords Surface rights owner Land ownership records Mineral rights ownerLand ownership records Drilling Rig Detection date/time Image analysisLocation Image analysis, g/c records Size Image analysis, g/c recordsSurface rights owner Land ownership records Mineral rights owner Landownership records Level of activity, Image analysis, sensor networks,inactivity, operation, g/c records status (idle, active) PipelineDetection date/time Image analysis infrastructure Location Imageanalysis, g/c records Length, width or size Image analysis, g/c recordsFluid type Image analysis, g/c records Surface rights owner Landownership records Mineral rights owner Land ownership records Bore sizeImage analysis, g/c records Flow rate Sensor networks, g/c recordsConnectivity Image analysis, g/c records Service road Detectiondate/time Image analysis Location Image analysis, g/c records Length,width or size Image analysis, g/c records Surface type Image analysis,g/c records Surface rights owner Land ownership records Mineral rightsowner Land ownership records Clearing Detection date/time Image analysisLocation Image analysis Length, width or area Image analysis Surfacerights owner Land ownership records Mineral rights owner Land ownershiprecords Trucks Detection date/time Image analysis Location Imageanalysis, GPS tracking (sensors or smartphones), roadside video camerasNumber Image analysis Type Image analysis Level of activity, Imageanalysis inactivity, operation, status (idle, active) Surface rightsowner Land ownership records Mineral rights owner Land ownership recordsTank Battery Detection date/time Image analysis Location Image analysis,g/c records Number of tanks Image analysis, g/c records Size of tanksImage analysis, g/c records Fluid level or volume Sensor networksSurface rights owner Land ownership records Mineral rights owner Landownership records Proppant store Detection date/time Image analysisLocation Image analysis, g/c records Size, volume or weight Imageanalysis, g/c records Proppant type Image analysis, g/c records Surfacerights owner Land ownership records Mineral rights owner Land ownershiprecords Drilling Detection date/time Image analysis reserve pit LocationImage analysis, g/c records Area or size Image analysis, g/c recordsNumber of sub-parts Image analysis, g/c records Fluid level or volumeImage analysis, sensor networks Surface rights owner Land ownershiprecords Mineral rights owner Land ownership records Frac spreadDetection date/time Image analysis Location Image analysis, GPS tracking(sensors, smartphones), roadside video cameras Number of pressure Imageanalysis, g/c records pumps Surface rights owner Land ownership recordsMineral rights owner Land ownership records Sand mines Detectiondate/time Image analysis Location Image analysis, g/c records Surfacerights owner Land ownership records Mineral rights owner Land ownershiprecords Producing Detection date/time Image analysis wells LocationImage analysis, g/c records Surface rights owner Land ownership recordsMineral rights owner Land ownership records Flare systems Detectiondate/time Image analysis Location Image analysis, g/c records Size Imageanalysis, g/c records Brightness, intensity or Image analysis, sensornetworks spectral content Surface rights owner Land ownership recordsMineral rights owner Land ownership records Level of activity, Imageanalysis, sensor networks, inactivity, operation, g/c records status(idle, active) Solar panel Detection date/time Image analysis mountsLocation Image analysis, g/c records Area Image analysis, g/c recordsNumber Image analysis, g/c records Type Image analysis, g/c recordsSurface rights owner Land ownership records Solar panels Detectiondate/time Image analysis Location Image analysis, g/c records Area Imageanalysis, g/c records Number Image analysis, g/c records Type Imageanalysis, g/c records Level of activity, Image analysis, sensornetworks, inactivity, operation, g/c records status (idle, active) Powerdelivery or Image analysis, sensor networks, capability g/c recordsSurface rights owner Land ownership records Electrical Detectiondate/time Image analysis substation Location Image analysis, g/c recordsSize Image analysis, g/c records Level of activity, Image analysis,sensor networks, inactivity, operation, g/c records status (idle,active) Power delivery or Image analysis, sensor networks, capabilityg/c records Connectivity Image analysis, g/c records Surface rightsowner Land ownership records Security fence Detection date/time Imageanalysis Location Image analysis, g/c records Length Image analysis, g/crecords Enclosed area Image analysis, g/c records Surface rights ownerLand ownership records Mineral rights owner Land ownership recordsBuilding Detection date/time Image analysis Location Image analysis, g/crecords Size or area Image analysis, g/c records Type Image analysis,building permit records, g/c records Surface rights owner Land ownershiprecords Mineral rights owner Land ownership records Cable systemDetection date/time Image analysis Location Image analysis, g/c recordsLength Image analysis, g/c records Type Image analysis, g/c recordsPower delivery or Image analysis, sensor networks, capability g/crecords Connectivity Image analysis, g/c records Surface rights ownerLand ownership records Wind energy Detection date/time Image analysiscollector Location Image analysis, g/c records Height above ground Imageanalysis, g/c records Type Image analysis, g/c records Surface rightsowner Land ownership records Level of activity, Image analysis, sensornetworks, inactivity, operation, g/c records status (idle, active)Meteorological Detection date/time Image analysis monitoring LocationImage analysis, g/c records system Height above ground Image analysis,g/c records Surface rights owner Land ownership records Level ofactivity, Image analysis, sensor networks, inactivity, operation, g/crecords status (idle, active) Construction Detection date/time Imageanalysis equipment Location Image analysis, g/c records Type Imageanalysis, g/c records Number Image analysis, g/c records Surface rightsowner Land ownership records Mineral rights owner Land ownership recordsLevel of activity, Image analysis, sensor networks, inactivity,operation, g/c records status (idle, active) Hydroelectric Detectiondate/time Image analysis reservoir or Location Image analysis, g/crecords forebay Area Image analysis, g/c records Water level or volumeImage analysis, sensor networks Water quality Image analysis, sensornetworks Connectivity Image analysis, g/c records Surface rights ownerLand ownership records Hydroelectric Detection date/time Image analysisintake structure Location Image analysis, g/c records Size Imageanalysis, g/c records Surface rights owner Land ownership recordsPenstock Detection date/time Image analysis Location Image analysis, g/crecords Number Image analysis, g/c records Length Image analysis, g/crecords Bore size Image analysis, g/c records Flow rate Sensor networksConnectivity Image analysis, g/c records Surface rights owner Landownership records Surge chamber Detection date/time Image analysisLocation Image analysis, g/c records Size or capacity Image analysis,g/c records Connectivity Image analysis, g/c records Surface rightsowner Land ownership records Hydroelectric Detection date/time Imageanalysis Power House Location Image analysis, g/c records Size or areaImage analysis, g/c records Connectivity Image analysis, g/c recordsSurface rights owner Land ownership records Level of activity, Imageanalysis, sensor networks, inactivity, operation, g/c records status(idle, active) Hydroelectric Detection date/time Image analysis TailraceLocation Image analysis, g/c records Size Image analysis, g/c recordsSurface rights owner Land ownership records Level of activity, Imageanalysis, sensor networks, inactivity, operation, g/c records status(idle, active)

In one example of the use of EI feature status attributes, the systemmay be configured to detect and measure flares and gas emissions as anindicator of oilfield activity and also as a measure and predictor ofoilfield well productivity when correlated to trailing public andprivate reports of oilfield well productivity. For example, the systemmay employ processing of aerial images to identify flare systems and todetect status attributes of flares or flare systems such as abrightness, an intensity, a spectral content, or a size, shortly after anew well is completed. The EI feature-level information 282 therebyobtained may be provided to other components of server 106 or to users190 or subscribers 195 of the online platform. In examples, the statusattributes of flares or flare systems may exhibit a correlation to oilor gas productivity and decline curves of a well when compared togovernment filings of oilfield well productivity such as may beavailable in a supplemental information source 270 such as publicrecords source 250. This correlation could then be used either by server106 or by users 190 or subscribers 195 of the online platform 180 toinfer the productivity and decline curve of new wells in the futurewhere a flare or emissions are detected but no government filings areyet available to indicate productivity. Amounts of flaring might alsoindicate shortages of pipeline capacity or insufficient economic basisfor transporting and selling oil and gas or might be used to estimatecost of operations for new wells in a particular area.

In addition to determining a current status of an EI feature attribute,EI feature determination manager 280 may be further configured to storeand build a history of feature attribute status, which may be stored inEI feature database 130, and to perform analysis on such history inorder to identify temporal changes or trends in EI feature status or topredict EI feature status. These may be subsequently reported to usersof the online platform or used as input to other functions performed bythe system.

Use of Supplemental Information

Different approaches may be used by the system to identify and classifyEI features from the aerial imagery. A basic approach is to detect andclassify each feature independently based purely on the image data (andprior training of an associated EI feature recognition model 216). Inother approaches, the ability, speed or accuracy by which the system isable to classify a feature may be enhanced through the use ofsupplemental information sources 270. Additionally, or alternatively,such approaches may also offer a reduction in the complexity ofimplementation of the EI feature recognition model 216, as thesupplemental information may allow the system to maintain a targetaccuracy of EI feature identification with less-complex orless-intensive image processing. Supplemental information sources 270may comprise an EI feature database 230 containing previously-obtainedinformation in the form of EI feature data records 240. Alternatively,or in conjunction, supplemental information sources 270 may comprise apublic records information source 250 comprising one or more publicrecords 260, for example as recorded by a government agency or otherauthoritative body.

By means of an illustrative example, the system may be arranged toconsult drilling permit records or drilling completion reports whenanalyzing images of global terrain and use this information to train anEI feature recognition model 216, for example by adjusting a structure,weights, coefficients, parameters, variables, thresholds orprobabilities used within the model, to increase the likelihood that anEI feature is correctly classified, or to reduce the time, computationalcomplexity or processing power required to reach a determination. Insome examples, as part of a training process, the system may consult asupplemental information source comprising oilfield drilling completionrecords and may use the information thereby obtained to determine thatan oilfield exploration site (one example of an EI site) exists at aparticular location, and that drilling commenced at the location at apast time D. The system may subsequently obtain and select as trainingimages, historical images of the portion of global terrain in which thedrilling operation is known to have been conducted. The images may havebeen captured at a range of times before or after time D. On the basisthat the EI site is known to have been operational at some point aftertime D, in some examples, the system may select historical images with acapture time later than time D, as training images on which to apply anEI feature recognition model 216 when operating in a training mode. Indoing so, the EI feature recognition model 216 may be updated tobetter-recognize EI features. In other examples, consultation ofhistorical oilfield drilling permit records may illustrate that arequest for a drilling permit for the same location was made at anearlier time P=D−Δ_(P). Further analysis of images of the same portionof global terrain, and captured at a time prior to time D, mayillustrate that an oilfield well pad was initiated or completed at timeWP=P±Δ_(WP) and that an oilfield frac-water pit was initiated orcompleted at time FP=P±Δ_(FP). The system may be configured to train oneor more EI feature recognition model(s) 216 to recognize correlations ofthis information with evidential proof (for example as obtained usingthe supplemental information) that drilling occurs.

More generally, supplemental or external information types may beobtained from EI feature database 230 or from public records informationsource 250 and may be used by the system to classify EI features.

FIG. 3B shows EI feature recognition model 216 in a working mode ofoperation. EI feature determination manager 280 or EI featurerecognition model 216 obtains one or more images from image selectionprocessing manager 212 or image selection storage 290, the imagesspanning a portion of global terrain. In some examples, images arecaptured by one or more aerial image sources, such as aerial imagesource 101. Working images 312 may comprise a single image or aplurality of images that were captured at substantially the same timeinstance. Additionally, or alternatively, working images 312 maycomprise an image time-sequence including a plurality of aerial imagesof the portion of global terrain, each aerial image of the pluralitytaken at one of a respective plurality of different image capture times.In some examples, EI feature recognition model 216 obtains one or morepieces of supplemental information from one or more supplementalinformation sources 270. Supplemental information sources 270 maycomprise EI feature database 230 in which one or more EI data records240 are stored. Additionally, or alternatively, supplemental informationsource 270 may comprise public records information source 250 in whichone or more public records 260 are stored. Examples of supplementalinformation sources and information are given in Table 1.

EI feature recognition model 216 may comprise artificial intelligenceprocessing such as a neural network or other machine learning algorithm.During working mode of operation, a structure, weights, coefficients,parameters, variables, thresholds or probabilities of EI featurerecognition model 216 (for example, as were generated, updated ormodified as a result of operation of EI feature recognition modeltraining manager 214 in training mode) are applied during processing ofworking images 312 in order, for example, to identify or recognize EIfeatures, to classify EI features as belonging to a given EI featuretype, or to determine a status attribute of an EI feature.

In some embodiments, EI feature recognition model 216 may include aconvolutional neural network (CNN). The CNN may include a plurality ofneurons, each of which may operate on a set of inputs x (for examplefrom other neurons) to produce an output y. In one example, the output ymay include the calculation by the neuron of a weighted sum of inputs xplus a bias variable b, such that y is a function of z=w·x+b. Here, xand w are multi-element vectors representing the inputs and weightsrespectively, whilst z, y and b may, in some examples, be single-elementreal or complex values. Calculation of y may further include anactivation function, such as for example a Sigmoid function, and whichmay be operable on z such that y=1/(1+e^(−z)). Numerous other knownactivation functions and basic calculations may be employed by neuronswithin a CNN-based image recognition model. EI feature recognition model216 may further include multiple layers, each layer comprising a groupof neurons that may be connected to a group of neurons in a previousand/or subsequent layer. Such layers may comprise an input layer, one ormore hidden layers, and an output layer. In examples, image data ispassed to the input layer, where it is processed by the group of neuronsassociated with the input layer, and the outputs thereby generated arepassed to the first of the hidden layers. Each successive hidden layermay process the set of inputs received from the previous layer in orderto calculate a set of outputs to pass to the next layer. The last hiddenlayer may calculate a set of outputs to pass to the output layer. Theoutput layer may calculate an output using the received set of inputsfrom the last hidden layer. In examples, the neurons in each layer mayeach perform a similar type of mathematical operation, for example, toperform a convolution function within a convolutional layer, or apooling (down-sampling) function within a pooling layer. Numerous otherlayer types are known in which different mathematical functions relatedto image-processing may be used. A neuron in a layer may be connected toone or more neurons of the previous layer and to one or more neurons ofa subsequent layer. At each neuron, each input connection may beassociated with a weight, for example an element of a weight vector w.In examples, during the training mode of operation, these weights, andother parameters, such as the neuron bias variables b, are recursivelyupdated within the EI feature recognition model 216, in order to reducean error at the model output and to thereby improve its recognitionperformance. The updated EI feature recognition model 216 may then beapplied to one or more working images to identify EI features.

In embodiments, application of EI feature recognition model 216 toworking images 312 results in an output from EI feature recognitionmodel 216 in the form of EI feature-level information 282. For example,EI feature-level information 282 may include information on.

-   -   A presence (or change thereof) of an EI feature;    -   A classification or type of an EI feature; or    -   One or more status attributes (or change thereof) associated        with an EI feature, such as its location or size.

In some examples, EI feature-level information 282 may also include oneor more confidence levels associated with the information or with acomponent of the information.

EI feature-level information 282 may be written to EI feature database230, for example in the form of EI feature data record 240, in orderthat it may be later used by the system or be provided to users 190 orsubscribers 195 of online platform 180. Optionally, EI feature-levelinformation 282 may only be written to EI feature database 230 if one ormore of the confidence levels associated with said information 282 meetsor exceeds a confidence threshold. In this manner, a reliability oraccuracy of EI feature-level information 282 held and stored within EIfeature database 230 may be maintained above a predetermined level.

EI feature recognition model 216 may be trained or otherwise optimizedfor a particular EI feature type or may be capable of recognizingmultiple EI feature types. In the event that the model is trainedtowards a specific EI feature type, a plurality of EI featurerecognition models 216 a, 216 b, . . . 216 n may be employed by thesystem, with each optimized for one of said feature types. In this case,a corresponding plurality of EI feature-level information 282 may bereturned, each of the plurality associated with a particular model andEI feature type.

In determining EI feature-level information 282, EI feature recognitionmodel 216 associated with a first EI feature type may utilizeinformation from one or more supplemental information sources 270. Suchinformation may relate to EI features at the same or a differentlocation to the identified EI feature of the first type and may relateto EI features of the same or a different type to that of first EIfeature type. In any of these cases however, EI feature recognitionmodel 216 may operate to combine visual information relating to apotential EI feature of the first type in the working image with thesupplemental information, which may be non-visual information, in orderto derive content within the EI feature-level information 282 that wasnot possible to derive using the visual information alone (that is, ifthe supplemental information was not provided to EI feature recognitionmodel 216). Thus, the combination of the supplemental information withthe visual information may improve the overall performance, accuracy orability of the system to identify EI features.

In the case that the information from the supplemental informationsource 191 relates to an EI feature type that is the same as the firsttype, this may be used to guide EI feature recognition model 216 andimprove its accuracy or false-alarm detection rate. For example, aworking image 312 may comprise a feature that closely resembles theshape and form of an oilfield frac-water pit. EI feature recognitionmodel 216 may identify the feature and a determine a size attribute.However, supplemental information may provide EI feature recognitionmodel 216 with knowledge of the size of other frac-water pits thatindicates to EI feature recognition model 216 that the determined sizeattribute of the identified feature is not in alignment with expectedvalues. As a result, EI feature recognition model 216 may either discardits identification of the feature or downgrade its confidence level thatthe feature is a frac-water pit.

By means of further oilfield-related example, and for the case in whichinformation from the supplemental information source 270 relates to anEI feature type that is different to the first type, EI featurerecognition model 216 may be trained to identify frac-water pits suchthat when applied to working image 312, EI feature recognition model 216determines with a first confidence level of e.g., 20% that a frac-waterpit exists at a first geographical location. The first confidence levelmay be relatively low due to a quality of the working image. However, EIfeature database 230 may comprise EI feature data record 240 thatindicates the presence of a previously-detected well pad within 200meters of the first geographical location at a confidence level of e.g.,75%. In light of the fact that frac-water pits and well pads tend to becollocated within a certain radius of one another, EI featurerecognition model 216 is able to significantly upgrade its confidence inthe presence of the frac-water pit at the first location to a secondconfidence level of e.g., 65%, and to subsequently write the EIfeature-level information 282 thereby obtained to EI feature database230. Descriptions of further related embodiments may be found withreference to, for example, FIG. 5A.

FIG. 3C depicts EI feature recognition model 216 a in both a workingmode of operation and in a training mode of operation. In both modes, EIfeature recognition model 216 a may be coupled to one or moresupplemental information sources 270, such as public records informationsource 250 (comprising one or more public records 260) or EI featuredatabase 230 (comprising one or more EI feature data records 240).

In the same way as previously described for FIG. 3B, in a working modeof operation, EI feature recognition model 216 of FIG. 3C is applied toworking image 312 and to information from one or more supplementalinformation sources 270 in order to generate EI feature-levelinformation 282. In doing so, new information (contained within EIfeature-level information 282) is generated which may be written backinto EI feature database 230 in the form of one or more EI feature datarecords 240.

As further illustrated by FIG. 3C, the new information thereby obtainedmay then be used by the system to train EI feature recognition model 216a in a training mode of operation. Thus, EI feature-level information282 (containing the new information) as generated by EI featurerecognition model 216 a in its working mode of operation may be writtento EI feature database 230 and subsequently used by EI featurerecognition model 216 a in its training mode of operation. Optionally,the new information may be used by EI feature recognition model 216 a inits training mode of operation only if the new information is deemed tobe sufficiently reliable (for example associated with a confidence levelabove a predetermined threshold). When in the training mode ofoperation, EI feature recognition model 216 a receives both a trainingimage 310 and the new information and EI feature recognition model 216 ais modified or updated by EI feature recognition model training manager214 accordingly as has been previously described. Updated EI featurerecognition model 216 a may subsequently be used in the working mode ofoperation,

FIG. 4 depicts one example of a method 400 for processing images toidentify EI features within aerial images of global terrain. In ageneral overview, method 400 may include receiving a working image 312comprising at least one aerial image of a portion of global terrain(step 410). Method 400 may include applying an EI feature recognitionmodel 216 to the working image 312 to identify information on a first EIfeature of a first type at a first confidence level (step 420). Method400 may include receiving information from a supplemental informationsource 270 on a second EI feature of a second type located in theportion of terrain (step 430). Method 400 may include updating theinformation on the first EI feature to a second confidence level basedon the first confidence level and the information on the second EIfeature (step 440). Method 400 may also include modifying, during atraining mode of operation, the EI feature recognition model 216 basedon at least one training image 310 of the portion of global terrain andon information on the first EI feature at the second confidence level(step 450).

Referring to FIG. 4 in more detail, method 400 may include receiving aworking image comprising at least one aerial image of a portion ofterrain (step 410). EI feature recognition model 216 is configured toobtain one or more training images 310, for example from image selectionprocessing manager 212 or image selection storage 290. In examples, anAPI is used to interface between EI feature recognition model trainingmanager 214 and image selection processing manager 212 or imageselection storage 290. In some embodiments, one or more training images310 are pulled from image selection processing manager 212 or imageselection storage 290. In some examples, one or more training images 310are pushed, for example at regular intervals, from image selectionprocessing manager 212 or image selection storage 290. In some examples,EI feature recognition model training manager 214 receives one or moretraining images 310 which span a portion of global terrain and whichwere captured by one or more aerial image sources, such as aerial imagesource 101. Training image 310 may comprise a single image or aplurality of images that were captured at substantially the same timeinstance. In examples, training image 310 may comprise an imagetime-sequence comprising a plurality of aerial images of a portion ofglobal terrain, each aerial image of the plurality taken at one of arespective plurality of measurably-different image capture times.

Method 400 may include applying EI feature recognition model 216 to theworking image to identify information on a first EI feature of a firsttype at a first confidence level (step 420). Examples of EI featuretypes are given in Table 2. In some embodiments, and in the case inwhich the EI feature recognition model 216 comprises a convolutionalneural network (CNN), the EI feature recognition model 216 may calculatea confidence level directly, for example based on the values of one ormore neuron layers within the CNN. Numerous different algorithms arepossible for such direct computation of a confidence level or confidencescore during image recognition and classification tasks in a CNN. Ingeneral, however, the confidence level may be based on a plurality ofneuron values (or a comparison, ratio or ‘distance’ therebetween)obtained within the EI feature recognition mode 216, for example withinthe last hidden layer, or an output layer of the CNN. In otherembodiments, confidence levels for identifying information on a first EIfeature from a working image may be derived by considering success ratesof previous determinations. For example, an EI feature recognition model216 determines that an EI feature of a given type exists based onanalysis of a working image. The working image captures aspects ofterrain that may be indicative of the development of the EI feature,even though the entire EI feature cannot be clearly seen in the workingimage. For example, EI feature recognition model 216 may determine thatan oilfield well pad exists in a portion of terrain based on imagecontent of a working image. An oilfield well pad is a surface site thatis comprised of a leveled, usually-rectangular area used to seatmachinery and equipment for drilling, completing, producing andmaintaining oil and gas wells. EI feature recognition model 216 maydetermine that what is captured in the working image represents a wellpad based on various attributes of well pads that can be detected in animage, such as the location and/or the area and size.

After the determination of the presence of this EI feature is made by EIfeature recognition model 216, other EI features may be subsequentlydetected by one or more additional EI feature detection model(s) 216through the examination of additional images captured at a later pointin time with respect to the determination of the oilfield well pad. Forexample, EI feature recognition model 216 may determine that an oilfielddrilling rig exists. If the drilling rig is determined to be present onthe well pad that was previously determined, then this subsequentdetermination of the oil rig validates the previous determination of thewell pad. This validation may be stored as a “success” of the first EIfeature recognition model 216 in determining the well pad feature at thestage of development that it did. In another example, one or moreadditional EI feature detection model(s) 216 may determine that anoilfield drilling rig does not appear on what was previously determinedto be an oilfield well pad. For example, a commercial building mayappear on that particular portion of terrain. This would indicate thatthe prior determination of the well pad feature was incorrect. Thisvalidation may be stored as a “failure” of the first EI featurerecognition model 216 in determining the well pad feature at the stageof development. Over time, the stored “success” and “failure”indications of the capability of the first EI feature recognition model216 to be able to accurate determine an EI feature at a given stage ofdevelopment of the terrain may be used to output a confidence level withthe determination of the feature. In this example, the confidence levelof the determination of the EI feature by EI feature determination model216 is based on a history of success and failure in determining such afeature at that stage of development.

Method 400 may include receiving information from a supplementalinformation source 270 on a second EI feature of a second type locatedin the portion of terrain (step 430). In some examples, supplementalinformation source 270 is an EI feature database 230 and the informationon the second EI feature is comprised within at least one EI featuredata record 240 stored within EI feature database 230. In some examples,the first EI feature type is the same as the second EI feature type. Inother examples, the first EI feature type is the different than thesecond EI feature type. In embodiments, the supplemental informationsource 270 is a public records information source 250 and theinformation on the second EI feature is comprised within at least onepublic record 260 stored within public records information source 250.Examples of supplemental information are given in Table 1.

In some embodiments, the information on the first EI feature or theinformation on the second EI feature comprises one or more of a presenceor a suspected-presence of an EI feature, a type of a known or suspectedEI feature, and/or a status attribute of a known or suspected EIfeature. Examples of status attributes are given in Table 3.

Method 400 may include updating the information on the first EI featureto a second confidence level based on the first confidence level and theinformation on the second EI feature (step 440). In some examples, theinformation on the second EI feature is associated with a thirdconfidence level, the method further comprising identifying theinformation on the first EI feature at the second confidence level basedon the first confidence level, the information on the second EI featureand the third confidence level.

Method 400 may also include modifying, during a training mode ofoperation, EI feature recognition model 216 based on at least onetraining image of the portion of global terrain and on information onthe first EI feature at the second confidence level (step 450). In someexamples, the training image is the same as the working image. In otherexamples, the training image is different than the working image. Inembodiments, an image resolution of the training image is higher than animage resolution of the working image. In examples, the use ofhigher-resolution images for training purposes may result in an EIfeature recognition model that is better-able to identify EI featuresfrom working images at lower resolutions. For example, a higherresolution training image may be used to train EI feature recognitionmodel 216 to recognize one or more key pixels in the lower resolutionworking images, or relationships between key pixels in the lowerresolution working images, that are indicative of the presence of an EIfeature. Thus, the EI feature recognition model 216 may be trained without-of-date higher resolution images, which are less expensive to obtainthan current high resolution images. The trained EI feature recognitionmodel 216 may then be applied to current lower resolution images, whichare also less expensive and/or easier to obtain than current highresolution images. This dual resolution approach provides a costsavings. In some embodiments, modifying the EI feature recognition model216 includes training a neural network to establish neural networkweights.

In some examples, the method further comprises applying modified EIfeature recognition model 216 to a further working image to identify afurther EI feature of the first EI feature type. In examples, modifyingEI feature recognition model 216 includes determining a pattern ofstages of EI feature development across an image time-sequence.

Joint Classification

Certain EI feature types may have a high correlation to other EI featuretypes, and a confidence level in the presence of one may spark astep-change in the confidence level of the other. The confidence levelupgrade may be sufficiently large that it exceeds a predeterminedconfidence threshold, thereby warranting inclusion of the EI feature bythe system in EI feature database 230 used by online platform 180.

Therefore, in a complementary or alternative enhancement, theidentification or classification of a particular first EI feature may bedetermined through consideration not only of the area of the image (orgroup of image pixels) in which the first EI feature itself isrepresented, but also by analyzing the image and/or otherwise obtaininginformation on one or more additional EI features that may be proximalto, interconnected-to, or otherwise related to the first feature. Whentaken collectively, the information on the one or more additional EIfeatures may help to confirm the detection or classification of thefirst EI feature. By means of example, a first EI feature in an oilfieldregion may be tentatively classified as a clearing with low probabilityor certainty. However, the system may also identify the presence of anew road connected to the clearing, and the appearance of a stockpile ofsand (proppant). As a result, and with knowledge that the presence ofthe clearing and the stockpile of sand are dependent upon a service road(in examples, EI feature relationship manager 220 maintains this kind ofknowledge and information), the system may determine with increasedcertainty that the feature of interest is indeed a clearing destined tobecome an oilfield well pad. Such a ‘joint classification’ technique mayoffer an improvement in detection and classification performance versusa model in which EI features are detected independently of each other,and wherein this collective relationship information (amongst related EIfeatures) is not exploited.

FIG. 5A shows a first embodiment of a system for identifying a first EIfeature using information on a second EI feature. EI feature recognitionmodel 216 a obtains one or more working images 312 from image selectionprocessing manager 212 or image selection storage 290 spanning a portionof global terrain and which were captured by one or more aerial imagesources, such as aerial image source 101. Working image 312 may comprisea single image or a plurality of images that were captured atsubstantially the same time instance. Additionally, or alternatively,working image 312 may comprise an image time-sequence including aplurality of aerial images of the portion of global terrain, each aerialimage of the plurality taken at one of a respective plurality ofsubstantially-different image capture times.

EI feature recognition model 216 a is applied to working image or images312 in order to generate an identification 510 of a first EI feature ofa first EI feature type at a first location within the portion of globalterrain and at a first confidence level. Identification 510 and theassociated first confidence level are provided as a first input to jointclassification processing manager 218.

As a second input, joint classification processing manager 218 isprovided with an identification 520 of a second EI feature of a secondEI feature type at a second location. The second EI feature type may beeither the same or different to the first EI feature type.Identification 520 is obtained from a supplemental or externalinformation source, such as an EI feature database 230 containing one ormore EI data records 240. In an alternative embodiment, the supplementalor external information source may comprise a public records informationsource. Optionally, identification 520 may comprise a third confidencelevel that is associated with the identification.

As a third input, joint classification processing manager 218 isprovided with relationship information 540 on a relationship between thefirst and second EI features from EI feature relationship manager 220.Such information may comprise any information indicative of acorrelation or relationship between the first and second EI features ofthe first and second EI feature types at the respective first and secondlocations. More generally, a relationship is defined as a correlationbetween two or more EI features of different EI feature types. By meansof example, relationship information 540 may comprise one or more of theexamples of information indicative of a relationship between EIfeatures, as given in Table 4.

TABLE 4 Examples of Relationships A functional or operational dependencebetween the first EI feature and the second EI feature. A functional oroperational dependence between the first EI feature type and the secondEI feature type. A surface-rights land ownership of the first and secondgeographic locations. A mineral rights land ownership of the first andsecond geographic locations. A statistical distribution of the distancebetween EI features of the first and second types. A conditionalprobability associated with the first EI feature and the second EIfeature. A conditional probability associated with the first EI feature,the second EI feature, and a distance therebetween. An association ofthe first or second locations to an identifier or location of anexisting or potential EI site. An association of the first or second EIfeatures to an identifier or location of an existing or potential EIsite.

Joint classification processing manager 218 operates on theidentification 510 of the first EI feature at the first confidencelevel, the identification 520 of the second EI feature (optionally atthe third confidence level) and on the relationship information 540 on arelationship (such as the examples of Table 4) between the first andsecond EI features in order to generate an updated (second) confidencelevel associated with the first EI feature. In FIG. 5A this is comprisedwithin identification 530 of the first EI feature at a second confidencelevel, which may in turn be written to and stored within EI featuredatabase 230 as an EI feature data record 240 for future use.Identification 530 may be one example of EI feature-level information282.

In an oilfield-related example, EI feature determination manager 280 maydetermine, using one or more EI feature recognition model(s) 216, that afrac-water pit is present in a portion of global terrain and that a wellpad is present in a portion of global terrain, and may determine adistance between the frac-water pit and the well pad (D_(FWP-WP)). Jointclassification processing manager 218 may also receive from EI featurerelationship manager, relationship information 540 comprising astatistical distribution of the distance between pairs of frac-waterpits and well pads that are known to be present at existing oilfieldsites. By using both the determined distance D_(FWP-WP) and the receivedrelationship information 540, joint classification processing manager218 may determine that the distance between the frac-water pit and thewell pad lies within a commonly-occurring range of the statisticaldistribution, and hence the two EI features are likely to be related tothe same oilfield site. In some embodiments, such a relationship of EIfeatures to the same EI site (in this example, an oilfield site) may bedetermined with some confidence level, for example based on acalculation using the supplied statistical distance information, or onthe past success and/or failure of such determinations. Thisdetermination may be used to update the confidence of the recognition ofone or both of the EI features (i.e. the frac-water pit and pit and thewell pad).

FIG. 5B shows a second embodiment of a system for identifying a first EIfeature using information on a second EI feature. The second embodimentis similar to the first embodiment of FIG. 5A though differs in that theidentification 520 of the second EI feature (along with its optionalassociated third confidence level) is provided to joint classificationprocessing manager 218 by a second EI feature recognition model 216 b,rather than by EI feature database 230. The second EI featurerecognition model 216 b may operate in a similar way to EI featurerecognition model 216 a in that it processes input images to identify EIfeatures or their status, for example usingartificial-intelligence-based image processing to do so. In one aspect,first (216 a) and second (216 b) EI feature recognition models may betrained or otherwise optimized to identify EI features of respectivefirst and second types, wherein the first and second types aredifferent. In another aspect, first (216 a) and second (216 b) EIfeature recognition models may be trained or otherwise optimized toidentify EI features of the same type.

In operation, one or more working images 312 from image selectionprocessing manager 212 or image selection storage 290 spanning a portionof global terrain are provided to both a first (216 a) and a second (216b) EI feature recognition model. In general, working image 312 maycomprise a single image or a plurality of images that were captured atsubstantially the same time instance. Additionally, or alternatively,working image 310 may comprise an image time-sequence including aplurality of aerial images of the portion of global terrain, each aerialimage of the plurality taken at one of a respective plurality ofsubstantially-different image capture times.

Whilst FIG. 5B depicts the same working image or images 312 beingprovided to both the first (216 a) and second (216 b) EI featurerecognition models, it shall be appreciated that in embodiments, aworking image or set of working images may be provided to the second EIfeature recognition model 216 b that is different to the working imageor set of working images that is provided to the first EI featurerecognition model 216 a.

In FIG. 5B, the first EI feature recognition model 216 a is applied tothe working image or images 312 in order to generate an identification510 of a first EI feature of a first EI feature type at a first locationwithin the portion of global terrain and at first confidence level.Identification 510 and the associated first confidence level areprovided as a first input to joint classification processing manager218.

The second EI feature recognition model 216 b is also applied to theworking image or images 312 in order to generate an identification 520of a second EI feature of a second EI feature type at a second locationwithin the portion of global terrain. Optionally, identification 520 maycomprise a third confidence level that is associated with theidentification. Identification 520 (and optionally the associated thirdconfidence level) are provided as a second input to joint classificationprocessing manager 218.

As a third input, joint classification processing manager 218 isprovided with relationship information 540 on a relationship between thefirst and second EI features from EI feature relationship manager 220,the nature of which has been previously described in relation to FIG.5A, and examples of which are given in Table 4.

Joint classification processing manager 218 operates on theidentification 510 of the first EI feature at the first confidencelevel, the identification 520 of the second EI feature (optionally atthe third confidence level) and on the relationship information 540 on arelationship between the first and second EI features in order togenerate an updated (second) confidence level associated with the firstEI feature. In FIG. 5B this is comprised within indication 530 of thefirst EI feature at a second confidence level, which may in turn bewritten to and stored within EI feature database 230 as an EI featuredata record 240 for future use. Identification 530 may be one example ofEI feature-level information 282. By means of example, EI featurerecognition model 216 a may be trained to identify proppant stores suchthat when applied to working image 312, EI feature recognition model 216a determines with a first confidence level of e.g., 35% that a proppantstore exists at a first geographical location. However, a second EIfeature recognition model 216 b may be trained to identify trucks, andmay determine the presence of a group of five trucks within 50 meters ofthe first geographical location at a confidence level of 40%. In lightof relationship information 540 from EI feature relationship manager220, that the transport of large quantities of proppant to oilfieldsites is often (for example, 70% of the time) performed by groups ofthree or more trucks operating in convoy, joint classificationprocessing model 218 may be able to significantly upgrade its confidencein the presence of the proppant store at the first location to a secondconfidence level of e.g. 80%, and to subsequently write thisidentification 530 thereby obtained (as one example of EI feature-levelinformation 282) to EI feature database 230.

In a general overview, FIG. 6 describes a method 600 for updating theidentification of an EI feature to a new confidence level based on aprevious confidence level, the identification of a second EI feature,and information on a relationship between the first EI feature and thesecond EI feature. Method 600 may include receiving a working imagecomprising at least one aerial image of a portion of terrain (step 610).Method 600 may include applying an EI feature recognition model to theworking image to identify a first EI feature at a first confidence levelaccording to image content of the working image (step 620). Method 600may include associating the first EI feature with a first EI featuretype and a first location within the portion of the global terrain (step630). Method 600 may include retrieving an identification of a second EIfeature in the portion of the global terrain (step 640). Method 600 mayinclude associating the second EI feature associated with a second EIfeature type and a second location within the global terrain (step 650).Method 600 may also include updating the identification of the first EIfeature to a second confidence level based at least on the firstconfidence level, the identification of the second EI feature, andinformation on a relationship between the first EI feature and thesecond EI feature (step 660).

Referring to FIG. 6 in more detail, in some examples, method 600 mayinclude receiving a working image comprising at least one aerial imageof a portion of terrain (step 610). In examples, the working image is alow-resolution aerial image or a portion of terrain. In some examples,the aerial image is accessible free of charge. In embodiments, theaerial image of the portion of terrain is updated periodically. Inembodiments, the aerial image is a satellite image. The satellite imagemay be updated periodically, for example each time the satellite passesover the portion of terrain. In some embodiments, the working image isreceived by image selection processing manager 212. Working image may beretrieved by image selection processing manager 212 from aerial imagestorage 105. Images received or retrieved by image selection processingmanager 212 may be stored in image selection storage 290, which maycomprise working images and training images. In examples, the workingimage includes an image time-sequence, the image time-sequence includinga plurality of aerial images of the portion of global terrain, eachaerial image of the plurality taken at one of a respective plurality ofimage capture times, and wherein applying the EI feature recognitionmodel includes recognizing a stage of EI feature development based onthe first EI feature in the working image.

In some examples, method 600 may include applying an EI featurerecognition model 216 a to the working image to identify a first EIfeature at a first confidence level according to image content of theworking image (step 620). In examples, EI feature recognition model 216a may be an AI model that has been trained to identify a specific EIfeature type. The method may include applying a plurality of EI featurerecognition models to the working image to identify the first EIfeature, each of the plurality of EI feature recognition models trainedto identify different EI feature types. In some embodiments, EI featurerecognition model 216 a may be applied to the working image in additionto one or more additional images from image selection processing manager212 or from image selection storage 290 to identify the first EIfeature. The one or more additional images that EI feature recognitionmodel 216 a is applied to may be selected by image selection processingmanager 212 to be images from the portion of terrain, from a portion ofterrain adjacent to the portion of terrain, or from the portion ofterrain at different times. In embodiments, the choice of the EI featurerecognition model may be made according to image content of the workingimage. In examples, the confidence level of the identification of thefirst EI feature may be chosen from a set of values. The set of valuesmay be a binary choice, wherein one value represents no confidence and asecond value represents full confidence (for example, the values 0 and 1respectively). The set of values may include a plurality of discretevalues, representing a scale of confidence between no confidence andfull confidence (for example the set of values may be [0, 1, 2, 3, 4,5], or [0, 0.2, 0.4, 0.6, 0.8, 1]) wherein a higher value representshigher confidence and the highest value represents full confidence. Theset of values may be represented by a continuum of values between twoendpoints, for example: confidence=[0,1], x∈R.

In some examples, method 600 may include associating the first EIfeature with a first EI feature type and a first location within theportion of the global terrain (step 630). Examples of EI feature typesare given in Table 2. In embodiments, EI feature recognition model 216 amay be generated by training a neural network to recognize EI featuresof a specific EI feature type. In some examples, a different EI featurerecognition model 216 a, 216 b, . . . 2116 n may be generated for one ormore specific EI feature types.

In some examples, method 600 may include retrieving an identification ofa second EI feature in the portion of the global terrain (step 640). Insome embodiments, identification of a second EI feature is retrieved byapplying a second EI feature recognition model 216 b to a working image.In examples, the identification of the second EI feature is retrievedfrom a supplemental information source 270. Examples of supplementalinformation sources are given in Table 1. In embodiments, thesupplemental information source is a public records information source250, and the supplemental information is a public record 260.

In some examples, method 600 may include associating the second EIfeature associated with a second EI feature type and a second locationwithin the global terrain (step 650). In some embodiments, different EIfeature recognition models 216 may be applied to the same image or setof images to determine EI features of different EI feature types fromthe same image or set of images. In some embodiments, the second EIfeature is a different EI feature type than the first EI feature.

In some examples, method 600 may also include updating theidentification of the first EI feature to a second confidence levelbased at least on the first confidence level, the identification of thesecond EI feature, and information on a relationship between the firstEI feature and the second EI feature (step 640). In embodiments, theidentification of the second EI feature includes its own confidencelevel, distinct from the confidence levels associated with the first EIfeature. The method may include updating the identification of the firstEI feature based on the identification of the second EI feature and itsconfidence level. In some examples, the confidence level of the first EIfeature may be updated to a new confidence level based on the confidencelevel of the identification of the second EI feature. In embodiments,updating the confidence level of the identification of the first EIfeature may be based, in part or in whole, on the first EI feature typeand the second EI feature type. In examples, updating the confidencelevel of the identification of the first EI feature may be based, inpart or in whole, on the location of the first EI feature and thelocation of the second EI feature, or in some embodiments on thedistance between the first EI feature and the second EI feature. EIfeature recognition model 216 may be modified in a training mode ofoperation, where the modification of the model is based on at least oneaerial image of the portion of terrain and on the updated identificationof the first EI features at the second confidence level.

In embodiments, the method may include updating EI feature database 230,for example either creating or modifying an EI feature data record 240.In some examples, the method may create, modify, or update an EI featuredata record 240 with a location of an EI feature and/or a type of any EIfeature. In examples, updating EI feature database 230 is contingentupon the confidence level of the identification of the first EI featuremeeting or exceeding a predetermined confidence threshold. The EIfeature data record 240 may include the confidence level of the EIfeature identification. The identification of the second EI feature maybe retrieved from EI feature database 230, which may be organized in EIfeature data records 240 comprising one or more pieces of information,for example presence, type, or status attribute.

In embodiments, the method may include identifying a status attribute ofthe first EI feature. Examples of EI feature status attributes are givenin Table 3. In examples, the status attribute of the first EI featuremay be stored in an EI feature data record 240 in EI feature database230. In embodiments, EI feature recognition model 216 identifies thestatus attributes of EI features.

The method may include providing EI feature-level information 282relating to the first EI feature to online platform 180, to be in turnprovided to user 190 and/or subscriber 195 of online platform 180, forexample over network 104.

Image Selection Processing

A primary task of the system is to detect and classify EI features usingimage processing within EI feature determination manager 280 or EIfeature recognition model 216. An appropriate set of images musttherefore be supplied to EI feature determination manager 280 or EIfeature recognition model 216.

A wide variety of considerations may play a role in selecting aparticular set of images on which EI feature determination manager 280or EI feature recognition model 216 is to be applied. However, as hasbeen previously described, EI feature recognition model 216 maygenerally be operated in training mode (operating on training images310) or in working mode (operating on working images). Hence, in someexamples of system 200, image selection processing manager 212 operatesto supply one or more training image 310, and/or one or more workingimage 312 to EI feature determination manager 280 or EI featurerecognition model 216.

FIG. 7A depicts a basic block diagram of image selection processingmanager 212, comprising in this basic example, image selection manager740. In this example, image selection manager 740 may select aerialimages from any available source to which it may be connected (eitherdirectly or via a network 104). Such available sources may includeaerial image source 101, aerial image storage 105, or image selectionstorage 290. In general, image selection manager 740 may group orcategorize aerial images according to certain criteria or according totheir suitability for a particular purpose. In one example, such as theexample shown in FIG. 7A, image selection manager 740 may categorizeimages as training images 310 or working images 312 and may providethese images to other components of system 200, such as for example, toEI feature determination manager 280 or EI feature recognition model216. In other examples, image selection manager 740 may categorize (andprovide to components of system 200) images according to a location, animage resolution, an image format, an image capture device and so forth.

As part of image selection processing manager 212, image selectionmanager 740 may also store selected images (optionally along with anycategorization thereof), for example in image selection storage 290 forlater retrieval or use. By means of example, image selection manager 740may receive via network 104, a first aerial image from aerial imagestorage 105, the first aerial image previously captured by aerial imagesource 101. Image selection manager 740 may categorize the first aerialimage as suitable for training mode purposes, and may store the image,for example in image selection storage 290 along with its categorizationas a “training image”. Such a categorization may be achieved by anysuitable means, for example, image selection manager 740 may storemetadata along with the image, the metadata comprising a suitable labelfor the determined category. Image selection manager 740 may furtherreceive via network 104, a second aerial image from aerial image storage105, the second aerial image also previously captured by aerial imagesource 101. Image selection manager 740 may categorize the second aerialimage as suitable for working mode purposes, and may store the image,for example in image selection storage 290 along with its categorizationas a “working image”. In general, image selection manager may assign oneor more than one categorization to an image. For example, an aerialimage may be categorized as suitable for both training mode and workingmode purposes and hence may be tagged with metadata including both a“training image” label and a “working image” label.

FIG. 7A shows a basic example of image selection processing manager 212.Other embodiments and methods are also possible in which additionalassociated processing may be advantageously employed. The area ofterrain to be analyzed by the EI feature recognition model(s) may bevery large, and the image recognition task may becomputationally-intensive, especially at higher image resolutions. Theprovision of sufficient computing resource to perform the task may beassociated with high operational or capital expenditures, thus from apractical standpoint, available processing power may be necessarilyconstrained and there is then a need to determine on which areas ofglobal terrain to focus the finite processing power that is available.

A further issue is that imagery suppliers may link the cost of aerialimages to the resolution of the image, the age of the image, or both.Thus, whilst the use of recent higher-resolution imagery is alwayspreferred in order to improve the timeliness and accuracy of the EIfeature recognition, the acquisition of such aerial imagery may carryhigh cost. Therefore, it may be beneficial to sparingly and selectivelyobtain high resolution aerial imagery only for those areas of terrainthat have a high probability of yielding new and useful EI feature-levelinformation from the application of detailed image processing. Such newinformation may comprise for example, the appearance or disappearance ofan EI feature, or a change in its status, such as a drop in a waterlevel of an oilfield frac-pit.

To address the aforementioned concerns, in an alternative embodiment andcomplementary method, additional auxiliary processing of the underlyingaerial image data may be used to assist with, steer or direct theprimary EI feature recognition task, for example to selectively focusavailable image processing power only on areas of terrain of particularinterest. Such auxiliary processing may be performed by image selectionprocessing manager 111 and may be run prior-to, or in-parallel-with theprimary EI feature detection and recognition task. In either case, theoutcomes of the auxiliary processing are used to control or obtain theset of images on which the EI feature recognition model(s) 216 operate.An example of such an embodiment is shown in FIG. 7B.

FIG. 7B depicts EI feature recognition model 216 operating on a workingimage or images 312, wherein the working image 312 is selected by imageselection processing manager 212. Image selection processing manager 212may operate by selecting images of portions of global terrain thatdemonstrate differences over time. By comparing images taken of portionsof global terrain taken at different times, the image selectionprocessing manager 212 can pre-select images that are more likely tocontain new features, such as new EI features. The image selectionprocessing manager 212 may detect differences that are indicative of EIsite development or construction.

Aerial image source 101 may be capable of providing images at differentresolutions. For example, high resolution images may be available andstored within a high-resolution repository 702, whilst lower-resolutionimages may be available and stored within a lower-resolution repository704, within one or more aerial image storage 105. Image selectionprocessing 212 may comprise a difference analysis processing manager 720that operates on a ‘comparison image set’ 770. In the example of FIG.7B, comparison image set 770 is shown comprising a plurality of aerialimages taken from lower resolution repository 704, though in general itmay comprise any set of aerial images (at any resolution) available, forexample from aerial image source 101, aerial image storage 105, anintermediate image repository such as high-resolution repository 702 orlower resolution repository 704, or from image selection storage 290.

Preferably, comparison image set 770 comprises at least a first image ofa portion of global terrain associated with a first time of imagecapture T₁, and a second image also spanning the first portion of globalterrain but associated with a second (later) time of image captureT₂=T₁+Δ. The difference analysis processing manager 420 operates tocompare the first image with the second image in order to determine oneor more geographical locations at which a difference in terrain contentappears to exist. Such difference analysis may be based on pixel valuesor any other image content data of the first and second images. A numberof image differencing techniques are known, and any of such techniquesmay be suitable for use by difference analysis processing manager 720.In a basic example, the first and second images may first be suitablyscaled and aligned in terms of position, resolution or both, such that afirst set of pixels in the first image and a second set of pixels in thesecond image correspond to the same geographical area of terrain. In afurther step, photometric or spectral analysis of the first and secondimages may be performed, and linear or non-linear adjustments made toone or both of the images to align or calibrate their brightness orcolor content. Once the images have been aligned in terms of scale,position, brightness or color, difference analysis processing manager720 may compute an image difference comparison (for example using asubtraction operation, or an XOR operation) between image content for aset of pixels in the first image and image content for a correspondingset of pixels in the second image.

The output of the comparison operation performed by difference analysisprocessing manager 720 may initially identify a large number ofdifferences or ‘hits’, hence it may be useful to reduce this set to asmaller number of focus locations (those of the most interest, ormost-likely to contain new information on EI features). To facilitatethis, the output from difference analysis processing manager 720 mayoptionally be passed (as is shown for the embodiment of FIG. 7B) todifference ranking or thresholding manager 730 for further processing inorder to determine a metric “M_(i)” representative of the importance orlevel-of-relevance of a particular difference having index “i”. In oneexample, M_(i) may be related to a magnitude or a scale of thedifference in terrain content. In another example, M_(i) may be relatedto a determined likelihood that the difference in terrain content isrelated to EI feature development. If such further processing is howevernot required, the output from difference analysis processing manager 720may be passed directly to image selection manager 740.

In deriving M_(i), attributes of the difference may be assessed bydifference ranking or thresholding manager 730, In some examples, saidattribute may be a size, a scale, a location, a color, a brightness, aluminosity, a shape, a proximity or relation to other image contentwithin the image difference comparison, or a geographical proximity toother known features within the portion of global terrain.

Whilst not shown explicitly in FIG. 7B, supplementary or external datasources, such as EI feature database 230 or public records informationsource 250 may also be used when determining M_(i). This allowsinformation such as previously-detected EI features and their locations,or drilling permit records, to also be factored-in as part of the M_(i)assessment.

By means of a general description, when deriving M_(i), a parameter orvalue v_(k,i) may be measured, calculated or otherwise assigned to thek^(th) attribute of difference “i”. For example, difference ranking orthresholding manager 730 may receive an image difference comparisonimage from difference analysis processing manager 720 in which a first(i.e. i=1) difference is identified comprising a contiguous region ofpixels that have values lying within a predefined color range. Such anidentified difference may be associated with a first (i.e. k=1) sizeattribute, for which a value v_(k,i) is measured as 134 pixels wide.Recognizing that some attribute types may have a stronger influence onthe importance or relevance level M_(i) than others, each may beassociated with a particular weighting factor wk. The overall importanceor relevance level M_(i) may then be calculated via a weighted sum asshown in the equation below:

$M_{i} = {\sum\limits_{k}{w_{k} \cdot v_{i,k}}}$

The set of M; for all detected differences may then be compared orranked by difference ranking or thresholding manager 730 in order todetermine those detected differences on which to focus. For example, thehighest-ranked “N” of these may be selected. In embodiments, the M_(i)values may be compared against a predetermined threshold, and thoseexceeding the threshold are selected. It shall be appreciated thatnumerous other methods of determining a subset of detected differenceson which to focus are also possible.

Image selection manager 740 may receive an output from either thedifference analysis processing manager 720 or (when employed) thedifference ranking or thresholding manager 730. Such output may comprisea subset of image differences (and/or their geographical locations) thathave been identified and selected. Image selection manager 740 operateson this output in order to select or obtain an appropriate working imageor images 312 that span one or more of the identified geographicallocations (where terrain differences have been identified) and on whichEI feature recognition model 216 is to be applied. The selected workingimage or images 312 represent one example of the selected image(s) 215shown in FIG. 2B. In a first example, image selection manager 740 mayselect working images 312 from comparison image set 770 or directly fromlower resolution repository 704. In a second example, image selectionmanager 740 may determine to select working images 312 from highresolution repository 702. If such high-resolution images are not yetreadily available to image selection processing manager 212 (forexample, fees to acquire them from an aerial image supplier have not yetbeen paid), image selection manager 740 may initiate a request 750 toobtain, via response 760, high resolution aerial images from highresolution repository 702, the high resolution images spanning one ormore of the identified geographical locations at which an imagedifference has been detected, and on which the EI feature recognitionmodel 216 is to be applied.

The output from difference analysis processing manager 720 or differenceranking or thresholding manager 730, although not shown explicitly inFIG. 7B, may also be used by the system for purposes other than imageselection. For example, the system may utilize the information ongeographical locations at which an image difference has been detected toinitiate the capture of additional aerial images spanning saidlocations. Additionally, or alternatively, the system may supply anotification that further follow-up inspection or a manual site visit atsaid locations is warranted. Such visits are costly in terms of time andman-power, hence the aforementioned difference analysis processing mayhelp to minimize unnecessary site visits and expenditure.

Working image or images 312 are then supplied to EI feature recognitionmodel 216, which analyzes the image in order to generate EIfeature-level information 282, optionally based also on supplementaryinformation obtained from EI feature database 230 (in the form of one ormore EI data records 240) or from public records information source 250(in the form of one or more public records 260). The EI feature-levelinformation 282 thereby derived may be written to EI feature database230 for storage and future use.

The description of FIG. 7B outlines processing that may be used by imageselection processing manager 212 to select one or more working image312. It should be appreciated that this is for illustrative purposesonly and that similar or related systems and methods may be employedwithin image selection processing manager 212 in order to selecttraining images 310, working images 312, or any other categorization ofimage. Image selection processing manager 212 may be further configuredto supply such selected images to other components of the system, suchas to EI feature determination manager 280, to categorize such selectedimages (for example as training images or working images) or to storesuch selected images, for example in image selection storage 290.

FIG. 8 represents a method 800 of selecting aerial images for imageprocessing to identify EI features. In a general overview, method 800may include retrieving a first plurality of aerial images associatedwith a first time of image capture, spanning a portion of global terrain(step 810). Method 800 may include retrieving a second plurality ofaerial images associated with a second time of image capture spanningthe portion of global terrain (step 820). Method 800 may includeidentifying one or more differences in terrain content between the firsttime of image capture and the second time of image capture (step 830).Method 800 may include identifying one or more geographical locations atwhich the respective one or more differences in terrain content havebeen identified (step 840). Method 800 may include selecting a set ofaerial images from the first, the second, or a third plurality of aerialimages based on the one or more geographical locations having adifference in terrain content (step 850). Method 800 may also includeapplying an EI feature recognition model 216 to the set of aerial imagesto identify at least one EI feature (step 860).

Referring to FIG. 8 in more detail, method 800 may include receiving afirst plurality of aerial images associated with a first time of imagecapture, spanning a portion of global terrain (step 810). The firstplurality of aerial images may be received as a group of images and/ormay be received individually and sequentially for further analysis bythe image selection manager 740. In one example, the first plurality ofaerial images may comprise at least a first image of a firstgeographical area within the portion of global terrain and a secondimage of a second geographical area within the portion of globalterrain. Both the first image and the second image may be associatedwith a first time of image capture T₁. The first time of image captureT₁ may be represented at different granularities or resolutions. Forexample, T₁ may be a time of capture that is accurate to one second, oneminute, one hour, one day, one week, one month or one year.

Method 800 may include receiving a second plurality of aerial imagesassociated with a second time of image capture spanning the portion ofglobal terrain (step 820). The second plurality of aerial images may bereceived as a group of images and/or may be received individually andsequentially for further analysis by the image selection manager 740. Inone example, the second plurality of aerial images may comprise at leasta third image of a third geographical area within the portion of globalterrain and a fourth image of a fourth geographical area within theportion of global terrain. Both the third image and the fourth image maybe associated with a second time of image capture T₂. The second time ofimage capture T₂ may be represented at different granularities orresolutions. For example, T₂ may be a time of capture that is accurateto one second, one minute, one hour, one day, one week, one month or oneyear.

In examples, a geographical location within the portion of globalterrain and spanned by the first or second images may also be spanned bythe third or fourth images, such that the geographical coverage of thefirst plurality of images overlaps with the geographical coverage of thesecond plurality of images.

Method 800 may include identifying one or more differences in terraincontent between the first time of image capture and the second time ofimage capture (step 830). In embodiments, image selection processingmanager 212 may compare a first comparison image from the firstplurality of images with a second comparison image from the secondplurality of images, wherein a particular geographical location isspanned by both of the first and second comparison images. Such an imagedifference comparison may be based on pixel values or any other imagecontent data of the comparison images. Various image differencingtechniques may be employed. For example, identifying differences interrain content may include adjusting a scale or a resolution of a firstcomparison image from the first plurality of aerial images to align witha scale or resolution of a second comparison image from the secondplurality of aerial images. In embodiments, image selection processingmanager 212 may align a first set of pixel positions in the firstcomparison image with a second set of pixel positions in the secondcomparison image such that the first and second sets of pixel positionscorrespond to the same geographical area. In some examples, imageselection processing manager 212 may compute an image differencecomparison between image content of the first comparison imageassociated with the first set of pixel positions and image content ofthe second comparison image associated with the second set of pixelpositions. In performing the comparison, image selection processingmanager 212 may perform photometric or spectral analysis of the firstand second comparison images, and linear or non-linear adjustments maybe made to one or both of the comparison images to align or calibrate,for example, their brightness or color content. Computation of the imagedifference comparison (for example, by a difference analysis processingmanager 720 within image selection processing manager 212) may be basedon, for example, a subtraction operation, or an XOR operation betweenimage content in the first and second comparison images. In embodiments,the difference in pixel values between at least one aerial image fromthe first plurality of aerial images and at least one aerial image fromthe second plurality of aerial images is representative of thedevelopment of an EI feature or EI site.

Step 830 may further include post-processing an image differencecomparison to identify differences. In embodiments, image selectionprocessing manager 212 may identify differences based on, for example, amagnitude, size or scale of image content within a previously-computedimage difference comparison, and optionally based on an assessment ofsaid image content against one or more thresholds. More generally, imageselection processing manager 212 may post-process an image differencecomparison to identify differences based on measuring or calculating avalue associated with an attribute of a candidate difference presentwithin the image difference comparison. In some examples, said attributemay be a size, a scale, a location, a color, a brightness, a luminosity,a shape, a proximity or relation to other image content within the imagedifference comparison, or a geographical proximity to other knownfeatures within the portion of global terrain.

In some examples, image selection processing manager 212 may determine alevel of relevance associated with an identified difference in terraincontent at at-least one of the one or more geographic locations. Inembodiments, the level of relevance is representative of one of amagnitude or scale in the difference in terrain content between thefirst time of image capture and the second time of image capture. Inembodiments, the level of relevance may be representative of adetermined likelihood that the difference in terrain content between thefirst time of image capture and the second time of image capture isrelated to EI feature development.

In embodiments, determining the level of relevance of the differencesobserved comprises determining a value for each of one or moredifference attributes associated with an identified difference interrain content and determining one or more weighting factors associatedwith each of the representative one or more difference attributes. Inexamples, the level of relevance of the identified difference in terraincontent is based on a sum over the one or more difference attributes, ofthe product of the value of each difference attribute and an associatedweighting factor. Processing to determine an importance or level ofrelevance of a candidate difference within a result of an imagecomparison, may further comprise communicating with an EI featuredatabase 230 or a public records information source 250 to retrieveinformation on previously-detected EI features and their locations.

In embodiments of method 800, image selection processing manager 212 mayselect images to store (for example in image selection storage 290), orto provide to other components of the system (such as EI featuredetermination manager 280 or EI feature recognition model 216) based onthe identified differences. In some examples, image selection processingmanager 212 may select a subset of the detected differences and base aselection of images thereupon. In general, image selection processingmanager may select images based on the determined level of relevance, aranking of a plurality of determined levels of relevance, and/or acomparison of one or more levels of relevance with a predeterminedthreshold.

Method 800 may include identifying one or more geographical locations atwhich the respective one or more differences in terrain content havebeen identified (step 840). In embodiments, image selection processingmanager 212 may determine a geographical location corresponding to aportion of image content within an image difference comparison, theportion of image content comprising an identified difference. Inexamples, the geographical location may be based on a known geographicalcoverage of the aerial images used to form the image differencecomparison, optionally in conjunction with projections or mappings ofpixel locations within the aerial images or within the image differencecomparison to geographical coordinates.

Method 800 may include selecting a set of aerial images from the first,the second, or a third plurality of aerial images based on the one ormore geographical locations having a difference in terrain content (step850). In some examples, image selection processing manager 212 selectsthe set of aerial images from the first plurality of aerial images, thesecond plurality of aerial images, or a third plurality of aerial imagesbased on a determined level of relevance, a ranking of a plurality ofdetermined levels of relevance, and/or a comparison of one or morelevels of relevance with a predetermined threshold. In some examples,the third plurality of aerial images has an image resolution that isdifferent from the image resolution of either the first or secondplurality of aerial images. The third plurality of aerial images mayhave an image resolution that is higher than the image resolution ofeither the first plurality of aerial images and the second plurality ofaerial images or both. The third plurality of aerial images may beobtained by the image selection processing manager 212 from any suitablesource, such as aerial image storage 105, aerial image source 101, orimage selection storage 290.

Method 800 may also include applying an EI feature recognition model 216to the selected set of aerial images to identify at least one EI feature(step 860). In examples, an image selection processing manager 212 mayprovide one or more selected aerial images 215 to an EI featuredetermination manager 280 or to an EI feature recognition model 216. Theselected aerial images 215 may comprise one or more training image 310,and/or one or more working image 312. In examples, EI featuredetermination manager 280 or EI feature recognition mode 216 may processthe selected aerial images 215 in order to determine EI feature-levelinformation 282. In embodiments, a server 106 comprising EI featuredetermination manager 280 or EI feature recognition model 216 mayprovide the determined EI feature-level information 282 to a user 190 ora subscriber 195 of online platform 180.

EI Site Status Based on Multiple EI Features

The foregoing has described the processing performed within the systemfor identifying and classifying EI features using one or more EI featurerecognition models and for determining the status of associated EIfeature attributes. Methods to improve or optimize the associatedprocessing have also been described.

In its raw or ‘stand-alone’ form, such EI feature-level information 282(as for example may be stored within the EI feature database 230 of FIG.2A) has significant value to users of online platform 180, who maysubsequently use and further process the information in a mannerspecific to their own needs.

In continued reference to FIG. 2A, the utility and value of the EIfeature-level information 282 may be further enhanced through additionalprocessing, for example by EI status determination manager 222 to form acomposite indication of EI site status 284. Such a composite indicationmay represent the activity or status at an EI ‘site’-level, the siteencompassing multiple individual EI features. In an oilfield-relatedexample, the EI site may be an oilfield drilling or fracturing site andthe multiple EI features may comprise a well pad, a frac-water pit, tankbatteries, and so forth.

FIG. 9A illustrates an oilfield-related example of such a set ofindividual EI features (912, 914, 916, 918, 920, 922, 924, 926) that maybe associated with an existing or potential EI site, such as an oilfielddrilling or hydraulic fracturing site 910. The system identifies, andstores information associated with EI features across the terrain andforms a determination as to which existing or potential EI site they maybe related to. This determination may be performed for example based onone or more of a geographical proximity or grouping of EI features, acommon surface-rights ownership, and a common mineral-right lease landownership. The information stored by the system relating to each EIfeature may further be associated with additional information such as aconfidence level 902 (for example, a level of certainty as to whetherthe identification of the EI feature at that location is accurate), orwith a status attribute 904 of the EI feature.

FIG. 9B illustrates a similar but solar-related example of a set ofindividual EI features (913, 915, 917, 919, 921) that may be associatedwith an existing or potential EI site, in this case, a solar powerstation 911. As was described for FIG. 9A, the information stored by thesystem relating to each EI feature may further be associated withadditional information such as a confidence level 903 or with a statusattribute 905 of the EI feature.

Early availability of a composite indication of EI site activity orstatus is of great interest to many different participants of the energyindustries, including operators, equipment suppliers, service providersand financial bodies. However, current methods of determining EI siteactivity are heavily reliant on word-of-mouth and formal public recordkeeping. These are often incomplete, unreliable or outdated and hencetheir usefulness is impaired. In an oilfield-related example, drillingpermits may often not be filed significantly in-advance of when drillingstarts, and some may even be filed after commencement of drilling. As aresult, methods currently in use provide information on EI site statusthat in fact lags actual status by several months.

In comparison, because the development of an EI site (such as anoilfield drilling site) takes some appreciable time, its early stagesare often visible significantly in advance of an associated permitfiling. Thus, by using and processing the collected feature-levelinformation (optionally in conjunction with other external datasources), the system of the present invention is able to substantiallyreduce the time taken to provide a composite indication of EI siteactivity or status at a given confidence level, for example, a 90%probability that oilfield drilling activity will commence within thenext 7 days.

Such an indication that oilfield drilling is likely to be imminent isjust one example of a composite indication of EI site activity or statusthat may be determined by the proposed system. Other examples ofcomposite indications of EI site activity or status are given in Table5.

TABLE 5 Examples of Composite Indications of EI Site Activity or StatusAn EI site has reached a particular stage of development. An oilfield EIsite has all necessary facilities to commence drilling. Drilling hascommenced on an oilfield EI site. Drilling has ceased on an oilfield EIsite. Hydraulic fracturing has commenced on an oilfield EI site.Hydraulic fracturing has ceased on an oilfield EI site. An EI siteexhibits an absence of activity. An EI site exhibits a commencement orrecommencement activity. A need for transport or infrastructureassociated with a resource at an EI site has been identified. A shortageor abundance of supplies, resource or equipment has been detected at anEI site. An EI site is non-operational and is not supplying energy to anelectrical power grid. An EI site is operational and is supplying energyto an electrical power grid.

By means of an oilfield-related example, the presence of a relativelylarge number of inactive rigs stacked in a storage yard may beindicative of ‘slack’ or excess capacity for drilling rigs, and hencereduced drilling costs may be forecast. Conversely, if all rigs areactive and the storage yards are empty, there may be a shortage of rigsand costs may rise. Similar logic may be applied to stores of resourcessuch as water or proppant.

Also, the relationship between features and company or well performancemay be used for competitive intelligence and process optimization. Forexample, the amount of time an oilfield rig or frac-spread is present ona site combined with lease operator identity might indicate theefficiency of an operator's business and return on capital. Or thenumber of pressure pumps in a frac-spread and size of a proppant storeand apparent water utilization when related to later-reported oil wellproduction figures for that location might indicate a more optimaldrilling and completion process that could be copied for better results.Examples of relationships are given in Table 4.

In the proposed system, each candidate composite indication of EI siteactivity or status is associated with a corresponding set ofcharacteristics or inputs (such as those of FIG. 9A or FIG. 9B). Thesystem has knowledge regarding each of these inputs and may associate aconfidence level 902 (FIG. 9A) or 903 (FIG. 9B) with each. For example,the system may associate the appearance of a new service road in anoilfield region with a pre-defined composite indication of EI siteactivity or status that the early stages of a developing oilfielddrilling site are present. The system may initially have only a 15%confidence level that the feature observed is indeed a road. However,other inputs or characteristics are also associated with the samecomposite indication, such as the appearance of a clearing, the absenceof any non-oilfield construction permits (such as for a home orindustrial building), the presence of a mineral lease, and a stockpileof sand. These characteristics are not independent, hence whilst thesystem may initially have only a low confidence level in each, it maydeduce that the likelihood of these features appearing within a givenradius or proximity of one another and within a given time span in anuncoordinated fashion is low, and hence may deduce a compositeindication, with high certainty, that the area is indeed a developingoilfield drilling site.

FIG. 9C depicts an embodiment of the system in which a compositeindication of EI site status 284 is determined by EI statusdetermination manager 222. A working image or images 312 is provided toboth a first (216 a) and a second (216 b) EI feature recognition model.The first EI feature recognition model may be trained or otherwiseoptimized to recognize EI features of a first type, and the second EIfeature recognition model may be trained or otherwise optimized torecognize EI features of a second type. EI feature recognition model 216a may output information 950 on the first EI feature at first locationand an optional first confidence level 952, whilst EI featurerecognition model 216 b may output information 960 on the second EIfeature at second location at an optional second confidence level 962.The information 950 on the first EI feature and information 960 on thesecond EI feature may optionally also comprise a status attribute of therespective first or second EI feature, such as the example representedby status attribute 904 of FIG. 9A or 905 of FIG. 9B. The information950 on the first EI feature, the information 960 on the second EIfeature, the first confidence level 952 and the second confidence level962 are examples of EI feature-level information 282 as shown in FIG.2B.

Optionally, EI status determination manager 222 may also receiverelationship information 980 on a relationship between the first andsecond EI features from EI feature relationship manager 220. Such arelationship may comprise any information indicative of a correlation orinter-relation between the first and second EI features of the first andsecond EI feature types at the respective first and second locations. Anon-exhaustive list of examples of relationship information 980 on arelationship between the two or more EI features is given in Table 4.

As a further option, EI status determination manager 222 may alsoreceive supplemental information 975 from a public records informationsource 250 (comprising one or more public records 260 relating to thefirst or second EI features or their locations) or from an EI featuredatabase 230 (comprising one or more EI data records 240 relating to thefirst or second EI features or their locations). Supplementalinformation 975 may comprise any of the supplemental or externalinformation types that have been previously described.

Various alternative approaches are possible for the way by which EIstatus determination manager 222 determines a composite indication of EIsite status using the available inputs 950, 952, 960, 962, 980 and 975.

In a first alternative, EI status determination manager 222 of FIG. 9Cutilizes a predetermined formula, algorithm, calculation or set of rulesor conditions that are associated with a candidate composite indication.These may be derived based on a composite indication configuration input970 comprising requirements that define what the candidate compositeindication is. For example, and in an oilfield-related context, onecandidate composite indication may be indicative of a developing EI site(in this case, an oilfield drilling or fracturing site) where a rig hasnot been installed. In this case, the requirements for the candidatecomposite indication may be that a number of EI features have beenidentified as present within a radius of 300 m of one another (such as awell pad, a service road and a frac-water pit), but that a rig has notbeen identified at the location of the well pad. In examples, compositeindication configuration 970 may be received from online platform 180,optionally via a network 104, or from an administrator 197, againoptionally via a network 104.

The inputs required by EI status determination manager 222 to executethe formula, algorithm, calculation, rules or conditions may be knownbeforehand and their values derived at least in part based on one ormore of the inputs 950, 952, 960 and 962. EI status determinationmanager 222 processes the feature-level inputs 950, 952, 960, and 962according to the composite indication configuration input 970 in orderto determine (for the candidate composite indication), an actualcomposite indication of EI site status in output 284. In one example,the composite indication 284 may be binary (for example true or false)and indicative of whether the conditions associated with the candidatecomposite indication (as defined by configuration input 970) have beenmet. In another example, the composite indication 284 may be associatedwith a confidence level, or with a value indicative of the degree towhich conditions associated with the composite indication have been met.

In a second alternative, EI status determination manager 222 of FIG. 9Cmay comprise Artificial-Intelligence-based (AI-based) processing, forexample, a neural network or other type of machine learning. TheAI-based processing may be suitably trained, for example, includingmethods and techniques as discussed above, to recognize and identify acorrespondence between the set of informational inputs (such as inputs950, 952, 960, 962, 980 and 975) and one or more candidate compositeindications of EI site activity or status (examples of which are givenin Table 5). Thus, an Artificial Intelligence model (which may beseparate to the first 216 a and second 216 b EI feature recognitionmodels used for feature-level detection from aerial images) is usedwithin EI status determination manager 222 and may be trained torecognize events or outcomes from a set of feature-level inputs thatcomprise feature-level information, feature-level status attributes, andpotentially external or supplemental information. Once the initialtraining is complete, the AI-based processing within EI statusdetermination manager 222 is then able to determine compositeindications of EI site activity or status in an automated manner. Themodel may be further trained and refined using further human supervisionor feedback from external, public, or other reliable informationsources. Similar to the first alternative, EI status determinationmanager 222 may receive requirements that define the compositeindication in composite indication configuration input 970. In oneexample, composite indication configuration 970 may be received fromonline platform 180, optionally via a network 104.

In a third alternative, as depicted in FIG. 9D, an advanced EI statusdetermination manager 224 may be employed which replaces the need forthe separate image-based processing (of EI feature recognition model216) and the EI status determination processing manager 222 of FIG. 9C.By means of further illustration, and with reference to FIG. 2A and FIG.2B, in some examples, advanced EI status determination manager 224 maybe used in lieu of separate EI feature determination manager 280 and EIstatus determination manager 222. Advanced EI status determinationmanager 224 is capable of generating a composite indication of EI sitestatus 284 directly from working image input 312 in conjunction withother optional inputs that may be available, such as supplementalinformation 975 and information on one or more relationships between EIfeatures 980.

In order to do so, advanced EI status determination processing block 224may employ artificial-intelligence based processing (such as a neuralnetwork or other machine learning algorithm) that is capable ofoperating on both image-based and non-imaged based informational inputsin order to generate the composite indication of EI site status 284.This differs from the system of FIG. 9C in that an intermediatedetermination of individual or constituent EI features (all relating insome way to the composite indication of EI site status) may not berequired. Thus, the advanced EI status determination processing 224 maybe trained to recognize the presence or status of an EI site as a whole(whatever its constituent parts or their development status) rather thanto recognize individual EI features.

In a similar manner to that employed for the first and secondalternatives of FIG. 9C, advanced EI status determination block 224 mayreceive requirements that define the composite indication in compositeindication configuration input 970. In one example, composite indicationconfiguration 970 may be received from online platform 180, optionallyvia a network 104.

FIG. 10 illustrates a method 1000 for processing images to determine EIsite status. In a general overview, method 1000 includes receiving aworking image including at least one aerial image of a portion of globalterrain (step 1010). Method 1000 may include applying a first EI featurerecognition model 216 a to the working image to generate information ona first EI feature at a first geographical location in the portion ofglobal terrain at a first confidence level according to image content ofthe working image (step 1020). Method 1000 may include applying a secondEI feature recognition model 216 b to the working image to generateinformation on a second EI feature at a second geographical location inthe portion of global terrain at a second confidence level according toimage content of the working image (step 1030). Method 1000 may alsoinclude determining a composite indication of an EI site status based onat least the information on the first EI feature, the information on thesecond EI feature, the first confidence level and the second confidencelevel (step 1040).

Describing FIG. 10 in more detail, method 1000 includes receiving aworking image including at least one aerial image of a portion of globalterrain (step 1010). In embodiments, an EI feature determination manager280 comprising a first EI feature recognition model 216 a and a secondEI feature recognition model 216 b may receive working image 312 from animage selection processing manager 212. Working image 312 may have beenselected by image selection processing manager 212 from amongst aplurality of available aerial images, for example aerial images from anaerial image source 101, an aerial image storage 105, or an imageselection storage 290, any of which may be accessed by image selectionprocessing manager 212.

Method 1000 may include applying a first EI feature recognition model216 a to the working image to generate information on a first EI featureat a first geographical location in the portion of global terrain at afirst confidence level according to image content of the working image(step 1020). In some examples, the first EI feature may be associatedwith a first EI feature type, some examples of which have beenpreviously described in this disclosure, for example in Table 2.

Method 1000 may include applying a second EI feature recognition model216 b to the working image to generate information on a second EIfeature at a second geographical location in the portion of globalterrain at a second confidence level according to image content of theworking image (step 1030). In some examples, the second EI feature maybe associated with a second EI feature type, some examples of which havebeen previously described in this disclosure. The first EI feature typeand the second EI feature type may be different in some examples. Insome examples, the second EI feature recognition model 216 b and thefirst EI feature recognition model 216 a may be the same EI featurerecognition model.

In embodiments, the information on the first EI feature 950 and theinformation on the second EI feature 960 further comprise a statusattribute that is representative of a status of the respective first EIfeature or second EI feature. Examples of status attributes that may beassociated with different EI feature types have been describedpreviously described in this disclosure, for example in Table 3. Somenon-exhaustive examples of such status attributes may include: adetection date or time; a location; an area or size; a length, a fluidlevel or volume: a number of sub-parts; a surface rights owner; amineral rights owner; a fluid or material type; a fluid or materialquality: a fluid or material color, a fluid or material attribute; aconnectivity, a brightness, intensity or spectral content, a powerdelivery or capability, a height above ground, a bore size, a flow rateand a level of activity, inactivity or operation.

Method 1000 may also include determining a composite indication of an EIsite status based on at least the information on the first EI feature,the information on the second EI feature, the first confidence level andthe second confidence level (step 1040). In embodiments, method 1000 mayinclude obtaining, based on the first geographical location or thesecond geographical location, supplemental information 975 from asupplemental information source 270 and determining the compositeindication of an EI site status 284 based further on the supplementalinformation 975. In examples, the supplemental information source 270 isa public records information source 250 or an EI feature database 230.Examples of composite indications of EI site status 284 related to an EIsite, such as an existing or potential hydraulic fracturing or oilfielddrilling site 910, are given in Table 5.

In embodiments, method 1000 may include obtaining, for example from anEI feature relationship manager 220 or a relationship manager storage292, information on relationships between EI features 980. Theinformation on relationships between EI features 980 may compriseinformation on a relationship between the first EI feature and thesecond EI feature. Some examples of relationships between EI featuresare given in Table 4. In embodiments, method 100 may include determiningthe composite indication of an EI site status 284 based further on theinformation on relationships between EI features 980.

In examples, determining the composite indication of an EI site status284 may be performed by an EI status determination manager 222 which maycomprise a processing function executing a pre-defined mathematicalfunction, for example operating on a set of inputs comprising one ormore of the information on the first EI feature 950, the firstconfidence level 952, the information on the second EI feature 960, thesecond confidence level 962. In examples, the determination of thecomposite indication of an EI site status 284 is performed by aprocessing function utilizing a neural network.

In examples, the composite indication of EI site status is associatedwith a binary value indicative of whether conditions associated with thecomposite indication have been met. The composite indication may beassociated with a confidence level or with a value indicative of thedegree to which conditions associated with the composite indication havebeen met.

In embodiments, a server 106 comprising EI status determination manager222 may provide the determined composite indication of EI site status284 to a user 190 or a subscriber 195 of online platform 180.

Retroactive Training

System 200 may further be configured to improve the accuracy ortimeliness by which it is able to identify EI feature-level status or EIsite status by utilizing the fact that EI site development occurs instages. In an oilfield-related example, at a developing oilfielddrilling or fracturing site, a road is first needed to bring trucks andmanpower into the area. A clearing may then be made, and a well paddeveloped. In parallel, a water pit may be constructed and filled, tankbatteries may be installed, and proppant stockpiled. In awind-power-related example, earth-moving and pile boring machinery mayfirst be used at a windfarm site in order to create tower foundationsand anchors. This may be followed by the arrival of specialized longtrucks to deliver segments of the towers, and large cranes to installthem. Once complete, work may commence on an electrical substationbuilding and overhead, ground level or subterranean cable systems may bebuilt. Whilst the stages of the above examples may not always besequenced in the same order, there is a manageable number ofcharacteristic patterns that can be identified and used to facilitate anearlier detection of the development of an EI site.

To exploit such sequencing, the system may be arranged to firstdetermine, at an eventual time “T_(x)” and with a high degree ofcertainty, that an EI site has been developed. This may for example bebased on identification of a number of associated EI features (forexample and in an oilfield-related context, a well pad, a water pit, arig or a service road) or may potentially be confirmed through anexamination of supplemental or external information (such as oilfielddrilling permit records or completion reports from a public informationsource).

The system may be further arranged to store a sequence of previoushistorical aerial images (over a period P) at the same location, forexample, from time T_(x)−P to time T_(x). The system may analyze thehistorical image sequence in order to determine a pattern ofdevelopment, including the sequence by which each of a plurality of EIfeatures appeared (or were first identified) and the associated times ortime spacings between these events. By associating a variety of suchpatterns known to correspond to the same EI site or the same eventualoutcome (e.g., commencement of drilling at an oilfield site), the systemmay be able to identify the development of future EI sites at a muchearlier stage.

To achieve this, system 200 may be configured such that an EI featurerecognition model 216 operates on a sequence of images (for example,comprised within selected images 215), rather than independently on animage-by-image basis. In this case, the historical image set known tocorrespond to a given outcome may be identified and selected for EIfeature recognition model training, in order that the model may improveits ability to recognize that outcome automatically in the future.Additionally, or alternatively, one or more EI feature recognitionmodels (for example, each trained to recognize a particular EI featuretype) may continue to operate on an image-by-image basis, and suitablepost processing is instead used to analyze the sequence by whichfeatures have appeared, and to correlate these against sequences knownto be associated with particular outcomes.

Training Using Multiple Image Resolutions

As previously described, obtaining aerial or satellite imagery may havea cost that is a function of both its age and its resolution. The mostexpensive imagery is often that which is both recent and high resolution(e.g., a resolution of 3 meters or less). Conversely, images that areeither old, or medium/low resolution (e.g. 10 meters or more), arelikely to be less costly or free.

System 200 may be configured to optimize its use of lower-cost imagery.In one approach, an artificial-intelligence-based model used for EIfeature detection and classification (such as EI feature recognitionmodel 216) may be trained using groups of two or more images that relateto the same area of terrain at the same time. At least one image of thegroup is at a high resolution, whilst another image of the group is at alower resolution. As the image data used for such training ishistorical, it may be obtained at a relatively low (or at no) cost.

Such training of EI feature recognition model 216 using both high- andlow-resolution image copies may allow it to improve its ability todetect and classify EI features using current (yet still inexpensive)lower resolution imagery without resorting to current (and expensive)high-resolution imagery.

For example, whilst learning or during training, EI feature recognitionmodel 216 may detect as a potential EI feature, image content related toa set of pixels on a low-resolution image, with a certain spatialrelationship to other EI features (e.g., as may be known via external,supplemental or previously-stored information). By supplying EI featurerecognition model 216 with a corresponding and contemporaneoushigh-resolution image of the same EI feature, the presence and type ofEI feature may be validated (or known with higher certainty), and thisvalidation may be used to improve the future ability of EI featurerecognition model 216 to recognize the feature using only low-resolutionimages. Thus, EI feature recognition model 216 is effectively trained torecognize EI features using low-resolution imagery with little or nohuman supervision.

Online Platform

As illustrated in FIG. 2A and FIG. 2B, the information obtained andgenerated by the system may be made available for users and subscribers(190, 195) of online platform 180 to access. Such information mayinclude EI feature-level information 282 which may be obtained from EIfeature database 230, or information 284 relating to a compositeindication of EI site activity or status. Whilst not explicitly shown inFIG. 2A or FIG. 2B for reasons of diagrammatical simplification,information from other sources, such as from public records informationsource 250 may also be provided to online platform 180 for users 190and/or subscribers 195 to access.

Users and subscribers (190, 195) of online platform 180 may comprise avariety of individuals, organizations or commercial entities. Each maybe interested in specific components of the information that has beencollected by the system as previously described. Users and subscribers(190, 195) may also have individual requirements regarding the accuracyor confidence level of the information that is reported to them. Bymeans of an oilfield-related example, a supplier of drilling equipmentand services may need to wait until there is sufficient informationabout the well pads associated with a particular oilfield operator (forexample their number, size, type, stage of development) to determine thescope and scale of the operation and hence whether the opportunity isone on which the supplier may wish to bid.

The proposed system is therefore capable of accepting and storing, foreach user and subscriber (190, 195) (or each individual, organization orcommercial entity) of online platform 180, a set of preferences orfilters that collectively allow the system to tailor the informationfeeds, reports or alerts provided to users and subscribers (190, 195).As an alternative or in addition to the information-feed approach, usersand subscribers (190, 195) of the system may log-in to online platform180 via network 104 to perform a targeted search for the informationthey require. The user preferences or search filters may include forexample:

-   -   EI feature types (or classifications) of interest (examples        given in Table 2);    -   EI feature status attributes of interest (examples given in        Table 3);    -   Composite indications of EI site activity or status of interest        (examples given in Table 5);    -   A confidence level that must be met before a piece of        information is reported; and    -   An age range that must be met before a piece of information is        reported.

Such options, preference or choices may be either selected from ageneralized preconfigured set of available options that are offered byonline platform 180 or may be bespoke-to (and potentially specified-by)users and subscribers (190, 195).

By means of example, FIG. 11 depicts a list of preconfigured reports1110 and a bespoke report 1120, both relating to composite indicationsof EI site activity or status in an oilfield context that may be ofinterest to users and subscribers (190, 195) of online platform 180. Inthe preconfigured case, a set of four available composite indicationreport types is shown from which users and subscribers (190, 195) mayselect. In the bespoke report case, users and subscribers (190, 195) mayspecify one or requirements related to a composite indication ofparticular interest to the users or subscribers (190, 195). Suchrequirements may be configured by users and subscribers (190, 195) bymeans of entering a logical expression (as shown), or by any othersuitable means such as via a graphical user interface of online platform180.

Once a preconfigured or bespoke composite indication report has beenselected or configured by the user or subscriber (190, 195), theassociated requirements are derived and used to retrieve the requestedinformation. This may comprise providing the requirements to otherfunctions within the system, for example, providing the requirements (ora parsed version of the requirements) to EI status determination manager222 and/or advanced EI status determination manager 224 as part of acomposite indication configuration 970. Whilst the above description ofthe configuration and selection of composite indication reports includesoilfield-related examples, the same approaches are equally applicable toother energy industries, such as solar, wind or hydroelectric power,with reports tailored instead to those specific types of EI sites and EIfeatures.

Display-of, or access-to the information provided by online platform 180may be performed via any suitable means such as an internet website, asmartphone application, an email application or a voice messagenotification service. The locations of EI features or situations ofinterest may be displayed on a map of the area together with anyassociated metadata or further details.

The reported information may be graphically or textually represented andmay further comprise interactive links allowing users 190 or subscribers195 to:

-   -   Obtain contact details or initiate communications with a related        party (e.g., a supplier, a potential customer or a rights-owner)    -   Flag an interest-in or ‘follow’ the identified EI activity, join        a group, receive further targeted updates, subscribe to a        related news feed    -   Obtain an expanded set of information available on the platform        at the selected location    -   Post an offer of products, services or equipment at the selected        site    -   Retrieve a list of posted offers for products, services or        equipment    -   Access permit, consent, or report data associated with the        location    -   Obtain an estimated relative surplus or shortage of supply        versus demand for a resource or service weighted by geospatial        proximity to the area of interest.

While the foregoing describes a particular set of techniques forprocessing aerial images to identify EI features, it will be understoodthat the information thereby obtained may be usefully employed by avariety of interested parties including energy infrastructure operators,suppliers of related resources, equipment or services, and financialanalysts or financial institutions.

The above systems, devices, methods, processes, and the like may berealized in hardware, software, or any suitable combination of these.The hardware may include a general-purpose computer and/or dedicatedcomputing device. This includes realization in one or moremicroprocessors, microcontrollers, embedded microcontrollers,programmable digital signal processors or other programmable devices orprocessing circuitry, along with internal and/or external memory. Thismay also, or instead, include one or more application specificintegrated circuits, programmable gate arrays, programmable array logiccomponents, or any other device or devices that may be configured toprocess electronic signals. It will further be appreciated that arealization of the processes or devices described above may includecomputer-executable code created using a structured programming languagesuch as C, an object oriented programming language such as C++, or anyother high-level or low-level programming language (including assemblylanguages, hardware description languages, and database programminglanguages and technologies) that may be stored, compiled or interpretedto run on one of the above devices, as well as heterogeneouscombinations of processors, processor architectures, or combinations ofdifferent hardware and software. In another aspect, the methods may beembodied in systems that perform the steps thereof and may bedistributed across devices in several ways. At the same time, processingmay be distributed across devices such as the various systems describedabove, or all the functionality may be integrated into a dedicated,standalone device or other hardware. In another aspect, means forperforming the steps associated with the processes described above mayinclude any of the hardware and/or software described above. All suchpermutations and combinations are intended to fall within the scope ofthe present disclosure.

Embodiments disclosed herein may include computer program productscomprising computer-executable code or computer-usable code that, whenexecuting on one or more computing devices, performs any and/or all thesteps thereof. The code may be stored in a non-transitory fashion in acomputer memory, which may be a memory from which the program executes(such as random-access memory associated with a processor), or a storagedevice such as a disk drive, flash memory or any other optical,electromagnetic, magnetic, infrared or other device or combination ofdevices. In another aspect, any of the systems and methods describedabove may be embodied in any suitable transmission or propagation mediumcarrying computer executable code and/or any inputs or outputs fromsame.

The method steps of the implementations described herein are intended toinclude any suitable method of causing such method steps to beperformed, consistent with the patentability of the following claims,unless a different meaning is expressly provided or otherwise clear fromthe context. So, for example performing the step of X includes anysuitable method for causing another party such as a remote user, aremote processing resource (e.g., a server or cloud computer) or amachine to perform the step of X. Similarly, performing steps X, Y and Zmay include any method of directing or controlling any combination ofsuch other individuals or resources to perform steps X, Y and Z toobtain the benefit of such steps. Thus, method steps of theimplementations described herein are intended to include any suitablemethod of causing one or more other parties or entities to perform thesteps, consistent with the patentability of the following claims, unlessa different meaning is expressly provided or otherwise clear from thecontext. Such parties or entities need not be under the direction orcontrol of any other party or entity and need not be located within aparticular jurisdiction.

It will be appreciated that the devices, systems, and methods describedabove are set forth by way of example and not of limitation. Absent anexplicit indication to the contrary, the disclosed steps may bemodified, supplemented, omitted, and/or re-ordered without departingfrom the scope of this disclosure. Numerous variations, additions,omissions, and other modifications will be apparent to one of ordinaryskill in the art. In addition, the order or presentation of method stepsin the description and drawings above is not intended to require thisorder of performing the recited steps unless a particular order isexpressly required or otherwise clear from the context. Thus, whileparticular embodiments have been shown and described, it will beapparent to those skilled in the art that various changes andmodifications in form and details may be made therein without departingfrom the spirit and scope of this disclosure and are intended to form apart of the disclosure as defined by the following claims, which are tobe interpreted in the broadest sense allowable by law.

1-17. (canceled)
 18. A computer-implemented method to identify EnergyInfrastructure (EI), the method to be carried out by at least oneprocessor executing computer instructions, the method comprising:receiving an image including at least one captured image of a portion ofterrain; obtaining gas emission information according to spectralcontent of the image; obtaining EI feature information on at least oneEI feature in the portion of terrain; identifying the at least one EIfeature according to the EI feature information; and associating the gasemission information with the at least one EI feature.
 19. Thecomputer-implemented method of claim 18, further comprising receivingfrom a supplemental information source, regulatory information relatedto the portion of terrain, wherein obtaining the EI feature informationincludes obtaining the EI feature information from the regulatoryinformation.
 20. The computer-implemented method of claim 19, whereinthe regulatory information includes at least one of oilfield permits andenergy infrastructure reports.
 21. The computer-implemented method ofclaim 18, wherein obtaining the EI feature information includes applyingan EI feature recognition model to the image to generate the EI featureinformation on the at least one EI feature.
 22. The computer-implementedmethod of claim 21, wherein: the image includes a second at least onecaptured image of the portion of terrain, and the EI feature recognitionmodel is applied to the second at least one captured image.
 23. Thecomputer-implemented method of claim 18, receiving from a supplementalinformation source, land data related to the portion of terrain, whereinobtaining the EI feature information includes obtaining the EI featureinformation from the land data.
 24. The computer-implemented method ofclaim 23, wherein the land data includes at least one of: pipeline maps,roadmaps, surface ownership records, lease records, and mineralownership records.
 25. The computer-implemented method of claim 18,further comprising receiving from a supplemental information source, GPSinformation related to the portion of terrain, wherein obtaining the EIfeature information includes obtaining the EI feature information fromthe GPS information.
 26. The computer-implemented method of claim 18,further comprising identifying at least one of an owner and an operatorof the at least one EI feature; and providing a report of the gasemission information to the at least one of an owner and an operator.27. The computer-implemented method of claim 18, wherein the at leastone EI feature includes at least one of: pipeline infrastructure,producing wells, flare systems, frac spreads, well pads, and drillingrigs.
 28. A computer-implemented method to identify EnergyInfrastructure (EI), the method to be carried out by at least oneprocessor executing computer instructions, the method comprising:receiving an image including at least one captured image of the portionof terrain; obtaining gas emission information according to spectralcontent of the image; obtaining EI feature information indicative of EIactivity in a portion of terrain; identifying the EI activity accordingto the EI feature information; and associating the gas emissioninformation with the EI activity.
 29. The computer-implemented method ofclaim 28, wherein obtaining EI feature information includes applying anEI feature recognition model to the image to generate the EI featureinformation indicative of EI activity.
 30. The computer-implementedmethod of claim 29, wherein: the image includes a second at least onecaptured image of the portion of terrain, and the EI feature recognitionmodel is applied to the second at least one captured image.
 31. Thecomputer-implemented method of claim 29, further comprising receivingfrom a supplemental information source, GPS information related to theportion of terrain, wherein identifying the EI activity includesidentifying the EI activity according to the EI feature information andthe GPS information.
 32. The computer-implemented method of claim 28,further comprising receiving from a supplemental information source, GPSinformation related to the portion of terrain, wherein obtaining the EIfeature information includes obtaining the EI feature information fromthe GPS information.
 33. The computer-implemented method of claim 28,further comprising identifying at least one of an owner and an operatorassociated with the EI activity; and providing a report of the gasemission information to the at least one of an owner and an operator.